Topaz AI stuff

From: william (WILLIAMA) 2 Oct 2020 14:33
To: ALL1 of 6
Probably just me, but anybody else playing with video enhancement? Got me a trial of this Topaz video enhance AI thing. Supposedly it isn't just a bit of sharpening filter, a touch of colour saturation adjustment, and some upscaling/remuxing. Supposedly it's the result of AI algorithms  trained on thousands of hours of before and after video images. Tried on some DVD samples from Spaced (another SD only release) and it's definitely better than I could manage with Avidemux. Quite impressive in fact.

The downsides are (1) after the trial it's quite an expensive product but more than that,  (2), it's bloody slow. A whole episode of Spaced on my desktop would have taken > 23 hours to process. 
From: ANT_THOMAS 2 Oct 2020 20:46
To: william (WILLIAMA) 2 of 6
Keep seeing either that or other similar things advertised. More than just a decent upscalling algorithm?

I'm guessing it hammers your CPU, what spec PC do you have these days?
From: william (WILLIAMA) 3 Oct 2020 09:11
To: ANT_THOMAS 3 of 6
PC Spec is oldish and then a bit further back. It's an i5 5690K (fairly respectable and not far off an i7 of the same generation) 8GB of DDR3, a GTX1050 (with only 2GB of DDR5), and a fairly decent Crucial SSD as the main drive.

Topaz is designed to run on an Nvidia GPU. They recommend a GTX 1080 with 8GB as the starting point*  which explains my extremely slow speeds. It will run on a CPU but slower still. 

As for what it is, I suppose really it is just a decent upscaling algorithm (or set of algorithms). They wave the "AI" flag, and I guess that means that they can tune the software by showing it variously degraded images at low resolution, followed by the high definition required result, so that it can do something like "best match" when dealing with real footage to decide how to enhance and upscale. The claim is that when the tuning is done with thousands of before and afters it can be very effective.

It has a bunch of profiles from which you pick one that matches the source video. For SD video, you're supposed to pick LQ (Low Quality). You then select the desired result which goes up to 8K. I've had some quite good results producing short samples. The biggest change came from picking the profile supposedly for CGI and applying it to an SD source, upscaling it to 1080P. Occasionally, it felt as though things were overdone: a "plastic" quality to skin tones and so on. But I have to admit it's a lot better than my attempts applying sharpening, denoise and smoothing filters with Avidemux. 

I've binned my experiments so far, but I'll do a couple of short sections and stick them up here as examples. Might take a while though!

*just looked at their help pages and they now say "RTX series or above with 6GB or more". They advise slightly different specs in different places, but you get the idea. 
From: ANT_THOMAS 3 Oct 2020 09:31
To: william (WILLIAMA) 4 of 6
Good that it's designed around GPU tech rather than just pure CPU horsepower.
From: Matt 3 Oct 2020 13:37
To: william (WILLIAMA) 5 of 6
Topaz is impressive, yes.

Some Internet folk are using it to upscale Star Trek DS9 to HD. The results are pretty dang good too:
From: william (WILLIAMA) 3 Oct 2020 18:42
To: Matt 6 of 6
Yes, I saw that. It looks interesting. One thing that did occur to me is that there are a couple of fellers who spent years, literally, enhancing Babylon 5 using more conventional techniques. I wonder how that would look if given the Topaz treatment. I have the current state of play which is good, but far from perfect.

If anybody's interested, here's a couple of short clips:

1) this is the opening of Spaced, episode 1, straight from the DVD. Technically it's 576P, but as it was shot on video tape for CRT televisions, that doesn't mean much.

2) this is the same clip fed through Topaz using the "Gaia-CG" profile with a target of MP4 1080P. 

The conversion took >2 hours and it's muted the sound a bit during the remux. If you do take a look, I recommend viewing at full screen as they can look a bit similar in a tiny little windowed player.
EDITED: 3 Oct 2020 18:47 by WILLIAMA