AI Neural Network image upscaling filters

[MusicallyInspired]

So this has been making the rounds in retro video game circles. It's been used with great effect for Doom sprite and texture upscaling and the Duke3D community has also been experimenting with it. I found a thread that featured upscaled backgrounds from Monkey Island 2 and the results are quite incredible (here's the thread with even more examples that are simply astounding). There are various models for filters available. One is waifu2x but that one makes things look only slightly better than your standard SuperEagle or hq2x filters. This one called ESRGAN (which uses a data set called Manga109 as a base) however....well, see for yourself.

Upscaled (no, seriously. this is an honest-to-god upscale without any post-editing):


Original:


And for the sake of comparison, here's the official MI2 Special Edition background:


The results are pretty fantastic. Here's another one from MI2:


Imagine seeing a filter like this generated in real-time in programs like ScummVM and DOSBox!

[MusicallyInspired]

A GIF comparison. I mean, just look at this. It's obviously not perfect but pretty darn close enough I'd say:

[MusicallyInspired]

Curse of Monkey Island:

[notbobsmith]

The comparison of the first two images are really quite impressive. I looks like it turned the background back into the original artwork. You can even see the brushstrokes.

[MusicallyInspired]

On the King's Quest 6 CD-ROM there were a collection of hi-res versions of backgrounds from the game in 512x365 resolution (game resolution is 320x200). Using this upscaling method I converted one to 2048x1460 and it looks phenomenal!!!

Upscaled w/ESRGAN (click to download at full res!):

Hi-res scan on KQ6 CD-ROM:

In-game low-res original background:

[Collector]

Now if this could be done in real time in DOSBox with the pixel perfect patch...

[MusicallyInspired]

Someone in the thread mentioned it took him 8 seconds to render an image with his new RTX video card. Give that 10 years (if that) and we'll have a silky smooth 60fps framerate out of that.

[Semi-Happy Partygoer]

Wow. This is incredible. I'll have to give it a go, though I'm not sure if my PC is fully compatible. I guess I'll just have to try and see.

[MusicallyInspired]

Here's a good starting point

[MusicallyInspired]

The Lord of the Dead (2048x1460):

[Collector]

Love it. Well, if someone wanted to take all of the graphics from KQ6 and pre-render them to remake the game it would be great. Of course it would need to be under the radar until after the release. What is the upper resolution that AGS will deal with?

[Semi-Happy Partygoer]

Here's a good starting point

Thank you for that. This is a bit beyond anything I've tried before, but I will make some time to give it a whirl. I always like learning new techniques.

[Tawmis]

These are amazing.

[Jules]

Wow those are so nice! The Lord of the Dead one is awesome.

[MusicallyInspired]

I took most of yesterday and wrote a script that extracts background resources from a Sierra game, converts to PNG, and upscales with ESRGAN. I'm doing all the Sierra games and uploading them to a Google Drive folder. I'll upload more as time goes on: https://drive.google.com/drive/folders/ ... sp=sharing

For PQ4 (and some screens in PQ3) I'm going to try a different model that's more accustomed and trained to restore low resolution live photos. It's coming out more like a painting, and while that's cool, I think it can do these better.

For QFG4, all the backgrounds are in pieces with foreground objects as different picture resources entirely so I'm going to take more time with that one and stitch them all together before upscaling.