This is an AI-generated animated music video of Aurora's song, DAYDREAMER, using Stable Diffusion Webui.
This video is for research and demonstration purposes only.
When I started animating I wanted to find a way to AI generate something in a more traditional animation style with lower frame rates. After some experimentation I wanted to challenge myself to create a full length AMV using only txt2img and prompt interpolation using this script:
https://github.com/EugeoSynthesisThirtyTwo/prompt-interpolation-script-for-sd-webui
No frames were edited in this video other than up scaling to 4k. No in-paint was used either. Every pose-to-pose animation was done through prompt interpolation. Although the results for some scenes and transitions are pretty jarring others came out looking very promising. I plan on releasing a guide with my workflow and thoughts on improving or optimizing the technique.
Thanks to the Stable Diffusion community for all the motivation and support! I love this community so much and hope we can all push boundaries together!
I want to see everyone's dream come to life!
______________________________________________________________
Original file: 768x448px (Still learning proper up-scaling)
Frames: ~2200
FPS: 10 (will look into frame interpolation techniques later)
Hours to complete: 10 hours total
Информация
Аниме: Original Animation
Музыка: Aurora - Daydreamer
Я конечно понимаю что интерполяция через подсказок выглядит не очень, кроме того 99% клипа состоит из лица женского персонажа, но тут просто невероятно крутая лирическая синхронизация что невозможно достичь через обычного монтажа клипа и темперамент трека очень хорошо выдержан. И еще концовка клипа повторяет начало клипа что тоже является плюсом к работе.