Topaz AI stuff

From: william (WILLIAMA) 2 Oct 2020 15:33
To: ALL1 of 22
Probably just me, but anybody else playing with video enhancement? Got me a trial of this Topaz video enhance AI thing. Supposedly it isn't just a bit of sharpening filter, a touch of colour saturation adjustment, and some upscaling/remuxing. Supposedly it's the result of AI algorithms  trained on thousands of hours of before and after video images. Tried on some DVD samples from Spaced (another SD only release) and it's definitely better than I could manage with Avidemux. Quite impressive in fact.

The downsides are (1) after the trial it's quite an expensive product but more than that,  (2), it's bloody slow. A whole episode of Spaced on my desktop would have taken > 23 hours to process. 
From: ANT_THOMAS 2 Oct 2020 21:46
To: william (WILLIAMA) 2 of 22
Keep seeing either that or other similar things advertised. More than just a decent upscalling algorithm?

I'm guessing it hammers your CPU, what spec PC do you have these days?
From: william (WILLIAMA) 3 Oct 2020 10:11
To: ANT_THOMAS 3 of 22
PC Spec is oldish and then a bit further back. It's an i5 5690K (fairly respectable and not far off an i7 of the same generation) 8GB of DDR3, a GTX1050 (with only 2GB of DDR5), and a fairly decent Crucial SSD as the main drive.

Topaz is designed to run on an Nvidia GPU. They recommend a GTX 1080 with 8GB as the starting point*  which explains my extremely slow speeds. It will run on a CPU but slower still. 

As for what it is, I suppose really it is just a decent upscaling algorithm (or set of algorithms). They wave the "AI" flag, and I guess that means that they can tune the software by showing it variously degraded images at low resolution, followed by the high definition required result, so that it can do something like "best match" when dealing with real footage to decide how to enhance and upscale. The claim is that when the tuning is done with thousands of before and afters it can be very effective.

It has a bunch of profiles from which you pick one that matches the source video. For SD video, you're supposed to pick LQ (Low Quality). You then select the desired result which goes up to 8K. I've had some quite good results producing short samples. The biggest change came from picking the profile supposedly for CGI and applying it to an SD source, upscaling it to 1080P. Occasionally, it felt as though things were overdone: a "plastic" quality to skin tones and so on. But I have to admit it's a lot better than my attempts applying sharpening, denoise and smoothing filters with Avidemux. 

I've binned my experiments so far, but I'll do a couple of short sections and stick them up here as examples. Might take a while though!


*just looked at their help pages and they now say "RTX series or above with 6GB or more". They advise slightly different specs in different places, but you get the idea. 
From: ANT_THOMAS 3 Oct 2020 10:31
To: william (WILLIAMA) 4 of 22
Good that it's designed around GPU tech rather than just pure CPU horsepower.
From: Matt 3 Oct 2020 14:37
To: william (WILLIAMA) 5 of 22
Topaz is impressive, yes.

Some Internet folk are using it to upscale Star Trek DS9 to HD. The results are pretty dang good too: https://www.extremetech.com/extreme/314653-remastering-deep-space-nine
From: william (WILLIAMA) 3 Oct 2020 19:42
To: Matt 6 of 22
Yes, I saw that. It looks interesting. One thing that did occur to me is that there are a couple of fellers who spent years, literally, enhancing Babylon 5 using more conventional techniques. I wonder how that would look if given the Topaz treatment. I have the current state of play which is good, but far from perfect.

If anybody's interested, here's a couple of short clips:

1) this is the opening of Spaced, episode 1, straight from the DVD. Technically it's 576P, but as it was shot on video tape for CRT televisions, that doesn't mean much.

2) this is the same clip fed through Topaz using the "Gaia-CG" profile with a target of MP4 1080P. 

The conversion took >2 hours and it's muted the sound a bit during the remux. If you do take a look, I recommend viewing at full screen as they can look a bit similar in a tiny little windowed player.
EDITED: 3 Oct 2020 19:47 by WILLIAMA
From: william (WILLIAMA)28 Feb 13:10
To: william (WILLIAMA) 7 of 22
Hmm

Got invited to a second free trial now that they've worked on Topaz for a while. Interested to see that they have some deinterlace and enhance options now. Not especially keen to deinterlace video as it runs fine through a TV, but it can be a pig to run enhance tools on interlaced video without deinterlacing it first. This  usually shags the quality thus defeating the object as you're trying to improve a much worse source file. So a program that does it in one hit sounds interesting.

So I plugged a 1hr 8 min episode of "The Singing Detective" in. 16 hours later - and some minutes - I had a pretty damn fine 1080P video from a none too special 576i source. Sadly the output file was 7 minutes longer than the input and the audio sync was all over the place.

The only cause of this that Topaz acknowledges is using a variable frame rate input (which this isn't) and so far my attempts to remux and resync the audio are way off.

Close, but no cigar.

And no way I'm spending the big bucks they want while it's still, to my mind, an alpha release.
From: ANT_THOMAS 2 Mar 08:39
To: william (WILLIAMA) 8 of 22
Can't things like ffmpeg do reasonable deinterlacing?

Yadif x2 seems to be a well regarded filter (as far as my knowledge of these things goes)
The Intel GPUs do some good Motion Adaptive and Motion Compensation filters in hardware. I assume nvidia would have some too, but not sure if they're supported via something like ffmpeg though.

Reason I know this is that I still watch TV at home via Tvheadend and Kodi. TVH feeds the broadcast video to Kodi which is interlaced, and Kodi does the deinterlacing. Usually requires an OK CPU or GPU hardware support.
From: william (WILLIAMA) 2 Mar 12:19
To: ANT_THOMAS 9 of 22
Yes, and I've used ffmpeg with yadif 1 and 2x plenty of times. I generally use it with Hybrid because although I have a stack of batch commands ready made in a big text file, I'm inclined to fiddle with the settings and Hybrid just makes it easier. The problem is that many of the files I play with are very poor quality to begin with: 70s and 80s TV series, even some from the 90s and later, taken from DVD. Somewhere, and I have much too little knowledge to say where, the transcoding process knocks just enough quality off to be counterproductive. I suspect if I was working with a decent quality 576i or 1080i original, it would be fine - but then I probably wouldn't bother deinterlacing anyway since most hardware decoders in TVs do a better job anyway and I wouldn't be looking to improve the quality.

I'm sure it's possible, but I've tried avidemux, avisynth, ffmpeg plus a whole load of free and commercial  deinterlacers and enhancers and I almost always end up with a large number of hours expended and a result that isn't a significant improvement on the original, if at all.

The really annoying part is that whereas I can buy absolute crap movies and series in every format up to including 4K (and maybe 8K soon) many of the classics of the big and small screens aren't available at all, and when they are it's in shitty DVDs, poorly produced. The BBC release of the Singing Detective isn't bad, it's just meh... a grainy, overcompressed, 576i. Potter wanted contrast between video inside shots of the hospital and film (some b&w) of outside and historical content. In the end it was all shot on film, so a decent remastering would be possible, always assuming the BBC haven't extended their usual level of tlc to the film, in which case it's probably rotted away somewhere. It won't happen. Potter's last couple of plays aren't even available new.

The good news, is that Topaz has a "basic" deinterlace/enhance profile that doesn't appear to change the video length and it's also faster: 11-12 hours per hour of video as opposed to 16 plus, with audio in sync. I ran the first episode through and it's very impressive, so I've just fired off a batch job with all 6 episodes. I'll post a couple of comparative samples when it's finished. A minor issue is that for some reason Topaz has decided to compress the audio to 96K max which may be a pain for some videos. No idea why they don't just pass it through as is and remux at the end. Still, it wouldn't take long to remux the output with the original audio myself.
From: ANT_THOMAS 2 Mar 13:20
To: william (WILLIAMA) 10 of 22
I guess with an already poor source you want to avoid multi stage compression to retain as much quality as possible
From: william (WILLIAMA) 2 Mar 13:50
To: ANT_THOMAS 11 of 22
Exactly. 
From: william (WILLIAMA) 3 Mar 22:08
To: william (WILLIAMA) 12 of 22
Couple of clips from the Singing Detective. I realised after I'd cut the bits out that I'd picked an area of better quality source video, but I think the difference/improvement is obvious.

Clip straight from the DVD

Clip after Topaz processing
From: ANT_THOMAS 4 Mar 13:12
To: william (WILLIAMA) 13 of 22
That's actually quite interesting. I paused both at a similar frame and it was fairly impressive that her badge has gone from pretty much totally impossible to read, to maybe possible to read. Significantly clearer and sharper
From: william (WILLIAMA) 4 Mar 13:55
To: ANT_THOMAS 14 of 22
I think some of the problems people have had with Topaz involve expectations. In essence each profile is just a type of filter, only with an enormously greater number of "if thens" than a standard "unsharp" or "denoise" etc. If you overdo a filter then not only do you overwrite real information with fake but cosmetically preferable information, you risk producing cosmetically undesirable results: ridiculously sharp edges where they shouldn't be, shiny, waxy, skin tones and loads of false detail. I saw an interesting blog where a photographer pushed an old landscape photo up to 4K using a Topaz photographic filter. The results were quite good, but looking closer, a road sign now had totally meaningless but clear and sharp "marks" where the words had previously been completely blurred. 

I stopped the processing at 1080P and also increased the default grain filter to add some noise back in. So Joanne Whalley's badge says "Miss C Mills Enrolled Nurse". I have a feeling that at 4K that might actually deteriorate. I'm pleased with that and will probably stop there, because it has enough clarity to look good on a big(ish) flatscreen telly. 

I have seen people online who are determined to get their ropey home-filmed VHS holiday epics up to 4K. That's quite literally true. It seems that if every single bit of the image isn't pin sharp then its unwatchable.
From: william (WILLIAMA) 5 Mar 13:51
To: william (WILLIAMA) 15 of 22
Trying the same settings on the DVD box set of Porterhouse Blue. This is a truly shabby Channel 4/Acorn DVD production. It was briefly released as a 2 DVD box set with 2 episodes per disk. This is now unavailable and the present production squashes everything onto 1 disk. Unbelievably, there are a couple of "library club" editions which are even worse. I managed to find the original 2 disk release on sale in Norway.

Ensemble cast of British character actors, Tom Sharpe novel with Malcolm Bradbury screen play, great cinematography and production values for broadcast, all shat on by Acorn DVD. Clumsy mastering, shaky, noisy, and way over-compressed.

Finding that the results aren't quite so good here. Because the source is so bad, I'm getting areas that look great, clear, sharp, faces for instance, in a sea of not-so-clear detail. Which doesn't look right. May have to redo them at 720P.
From: william (WILLIAMA) 8 Mar 11:21
To: william (WILLIAMA) 16 of 22
Weird user forum, by the way. Looks to be custom made, although it's fairly normal at first glance. You don't join the forum*, you gain membership by using it. Sounds reasonable but since there are no obvious instructions it's really confusing. The categories for posting are muddled up in different places and although you aren't told when you start using the forum, you can't post to it straight away. It seems that, without exception, people waste their first hours trying to find what the fuck's going on. People turn up having spent £100s on the various products and feel as though they're being treated like shit. 

I spent ages wondering what I was doing wrong because although, for some reason, I could post in a section on photographic filters (where I had nothing to say and no particular interest) the new post button for Video enhancement was (is) greyed out.

I found a bundle of posts in a general section which basically asked WTF? Somebody said they'd finally realised that they had to post stuff where they could, to be allowed to post where they actually wanted to. I (was able to) comment on this that it seemed a bizarre idea, to have to comment on stuff I had no interest in or knowledge about. This was pounced on by a "moderator" with a patronizing reply:
 
Quote: 
That is what spam control is all about. Typically spammers go away if they can’t post immediately. It may be inconvenient for you but works for the site. Just raise a post in say Topaz Products and a moderator will catch it for you.


And there's the main issue. Very enthusiastic mods. At least one, apparently, has fun deleting and editing posts he doesn't like. I've been using online forums since they were known as bulletin boards and I've never seen one quite so badly thought through.

*they use the account set up when you get a download of a product
EDITED: 8 Mar 11:23 by WILLIAMA
From: Peter (BOUGHTONP) 8 Mar 14:14
To: william (WILLIAMA) 17 of 22
> It may be inconvenient for you but works for the site.

I think that sums up the ethos Apple/Google/etc have brought - so much software isn't about the users any more, it boils down to being another mechanism of control.


> Weird user forum, by the way. Looks to be custom made, although it's fairly normal at first glance.
> You don't join the forum*, you gain membership by using it. Sounds reasonable but since there are
> no obvious instructions it's really confusing. The categories for posting are muddled up in different
> places and although you aren't told when you start using the forum, you can't post to it straight away

That had me thinking "I bet they're using Discourse", and I was right.

Discourse is what you get when the imbeciles behind StackOverflow think they know what they're doing, instead of realising their success there was mostly due to being less shit than Experts Exchange, and in spite off all the dumb hurdles they added, not because of them.

EDITED: 8 Mar 14:38 by BOUGHTONP
From: william (WILLIAMA) 8 Mar 14:47
To: Peter (BOUGHTONP) 18 of 22
You're 100% right. That's probably why there are lively active threads on "why the forum is so bad" (so long as the particular mod I mentioned allows them to continue) and so few on the software itself.



 
From: CHYRON (DSMITHHFX)11 Mar 16:44
To: Peter (BOUGHTONP) 19 of 22
I suspect a lot of the activity is down to performative busywork by really bored, underemployed it hacks.
From: william (WILLIAMA)12 Mar 16:48
To: CHYRON (DSMITHHFX) 20 of 22
I suspect you're right. It's not exactly a bustling hub of enthusiasm anyway. It should be, considering that Topaz products have created quite a stir in photographic and video circles in the last year, but I think the forum software is really stifling discussion. Plus, any time things kick off, the arsehole mod steps in, declares the discussion "off topic" or "not relevant" and either edits posts or cancels the thread altogether.

The video categories, in particular, are dire. It's a product that is clearly designed for high-end NVIDIA GPUs. It will run (just) on a very high-end CPU but may take multiple times longer. Yet the discussion has far too many posts along the lines of "it takes 60 hours to do 10 minutes of vid on my Matrox Mystique, what's wrong?" and "will this work on my Atom-powered notepad?"