Project Starlight - Topaz Labs

[thecyanray]

Topaz Labs released a major new model very recently. They claim that it "transforms low-resolution and degraded video into HD quality". Some descriptors from them:

It's the first-ever diffusion AI model for video enhancement and the only one to achieve full temporal consistency, ensuring seamless motion from frame to frame. Users can access the research preview online or in Video AI.

Take almost any video to 4K: Project Starlight uses diffusion technology for the clearest AI video enhancement to date. It upscales, enhances, denoises, de-aliases, and sharpens video—without any sliders or manual adjustments.

Each frame restored by Project Starlight is created by analysing hundreds of frames surrounding it. This creates a smooth quality that has never been achieved in AI footage restoration before, called temporal consistency.

While existing video enhancement models utilize GAN technology, the switch to diffusion is one key to Project Starlight's major quality boost. Unlike existing GAN models, Project Starlight’s models have a great understanding of semantics, what objects actually are, and how motion and physics work—so they can make things look more natural with less to work with.

In 2023, we tried improving quality on our video models while holding speed and size constant, but this approach was ineffective. While later iterations (Rhea, RXL) do offer improvements over earlier versions (Proteus, Artemis), you’ve probably noticed that the changes have been getting more incremental. So in 2024, we asked the question: “How much video quality could we achieve if we didn’t care about speed or size?” The result is Project Starlight. It requires huge VRAM and currently takes 20 minutes to process 10 seconds of footage, which we know is sort of ridiculous. The results, however, are truly mind-blowing - an accomplishment which was quite challenging just by itself.

As stated above, it currently takes twenty minutes to process ten seconds of footage using server-grade hardware (apparently H100 GPUs, which are specifically designed for AI workloads and go for around 30K each). It's therefore only currently available as an online preview operating on a credit model (it would currently cost over $1,500 worth of credits to enhance a 1h 30m film), unless you happen to have a H100 GPU/cluster and are willing to pay an enterprise subscription, then they can allow you to process locally. They will be working on lighter models in the near future for consumer local processing:

In 2025, we’ll focus on optimizing Starlight in a similar way. We’ve only focused on quality for this release, and we wanted to get it usable as soon as possible. You can expect smaller and less expensive models in the near future, based on the same technology that we’ve developed with Starlight. In the meantime, we’re initially pricing this model at cost to help make it more accessible.

I've enhanced two short clips of moderately degraded quality with Starlight: One from Spielen wir Liebe (1978) and one from Mords Pas On t'Aime (1976)


Spielen wir Liebe: [Code] Mords Pas On t'Aime: [Code] Before/After comparison slider for Spielen wir Liebe: [Code] Before/After comparison slider for Mords Pas On t'Aime: [Code] Personal notes: Some dust/scratches were picked up on an removed, but it's clear that you would need a program specialised for that task to ensure complete/satisfactory removal. The enhancement is a bit aggressive for me, especially in terms of faces, but it does do a great job with the background; I think it would look great with some grain introduction from Topaz under the Gaia model (low invasive). Overall, promising results, I'm interested in your thoughts!

[ghost]

Yes, I've read about the Topaz Starlight project. I didn't test it because you only can run it in the cloud and it isn't for free (even if you paid for Video AI).

The results are way better than the models you can run locally, but a bit agressive to my opinion. Sometimes the details are too sharp for my taste.

Let's see what the future will bring...

[thecyanray]

How to access Starlight?

As of today... Through this site:
[Code] Or through the Topaz VEAI application.

There are two options available:

1. Free rendering. Three drawbacks are that you can only render 300 frames at a time; you only get three renders; your renders are not private, Topaz can review them for training.

2. Rendering with credits. You can render five minutes at a time (can split up a file and recombine). Your renders are private.

A local alternative is STAR. Some have speculated that Topaz Labs have actually claimed this model as their own, but they have denied this. You will need a very powerful GPU (like a 5090 or RTX 6000 Ada) to make real use of it.
[Code]

[ghost]

A local alternative is STAR. Some have speculated that Topaz Labs have actually claimed this model as their own, but they have denied this. You will need a very powerful GPU (like a 5090 or RTX 6000 Ada) to make real use of it.

Interesting.

I only have a RTX 4090 mobile.

[Unigami]

A local alternative is STAR. Some have speculated that Topaz Labs have actually claimed this model as their own, but they have denied this. You will need a very powerful GPU (like a 5090 or RTX 6000 Ada) to make real use of it.

Here! That's why I don't trust AI and don't bother. Because It is both expensive and requires power. -_-

Thecyanray,
Which movie is your avatar from? I was curious.

[thecyanray]

Which movie is your avatar from?

Piccole Labbra (1978)

[Rich_Visiting]

Damn, I was just about to start a thread on Starlight. But it's a cloud service, so how to get a version that can run on local PC? I see many Topaz customers are already pissed that its on cloud btw i don't believe the bull that it requires too much power for local computers - just a way to make $$$$$$$$$$$$$$$$

[ghost]

so how to get a version that can run on local PC?

That won't be possible. And I guess you'd need a graphic card with 500 GB VRAM.

[Rich_Visiting]

so how to get a version that can run on local PC?

That won't be possible. And I guess you'd need a graphic card with 500 GB VRAM.

current AI is so waxy I've tried many models (Topaz AI Video) but struggle to find one that is all inclusive to all content.

[Rich_Visiting]

A local alternative is STAR. Some have speculated that Topaz Labs have actually claimed this model as their own, but they have denied this. You will need a very powerful GPU (like a 5090 or RTX 6000 Ada) to make real use of it.
[Code]

Thank you!! I should really learn to read the whole thread. Is there a simple guide to this? YT gives nothing.