Release Notes: Gemini's multimodality

 

"Ani Baddepudi, Gemini Model Behavior Product Lead, joins host Logan Kilpatrick for a deep dive into Gemini's multimodal capabilities. Their conversation explores why Gemini was built as a natively multimodal model from day one, the future of proactive AI assistants, and how we are moving towards a world where "everything is vision." Learn about the differences between video and image understanding and token representations, higher FPS video sampling, and more."

List of Video Generation Tools

# List of Video Generation Tools [Source: s-marke]

## USA

* Runway ML: https://runwayml.com/

* Pika: https://pika.art/home

* Luma Labs Dream Machine: https://lumalabs.ai/dream-machine

* OpenAI Sora (not published): https://openai.com/index/sora/

* Meta Make-A-Video: https://makeavideo.studio/

* Adobe Firefly Video: https://blog.adobe.com/en/publish/2024/09/11/bringing-gen-ai-to-video-adobe-firefly-video-model-coming-soon)

* Kaliber AI: https://kaiber.ai/

* MidjJourney (model currently trained): https://www.midjourney.com/

* Warpvideo: https://warpvideo.ai/

https://leonardo.ai/

## China

* Kling AI: https://klingai.io/

* Minimax AI: https://minimax-ai.org/ or https://hailuoai.video

* Vidu: https://www.vidu.studio/

* Qingying

## Other Countries

* Black Forest Labs: (DE, funded by USA. Model is currently trained) https://blackforestlabs.ai/

* LTX Studio (Israel): https://ltx.studio/

* Haiper AI (UK): https://haiper.ai/

* Leonardo AI (Australia): https://leonardo.ai

# Open Weight Models

* Open Sora: https://github.com/hpcaitech/Open-Sora

* Cog Video: https://github.com/THUDM/CogVideo

* Pyramid Flow: https://github.com/jy0205/Pyramid-Flow

* Stability AI (video diffusion model, unsure, which country):  https://stability.ai/

* Mochi 1 (commercial and free to use): https://mochi1.online/, https://huggingface.co/genmo/mochi-1-preview

* HunyuanVideo

# videos

* The office: Oval Edition: https://www.youtube.com/watch?v=p-nAGeaif0U

* Dior - The Ascendance: https://x.com/CuriousRefuge/status/1844068172623905069

* Gorilla Story: https://x.com/rainisto/status/1843296900599996742

* Famous portraits brought to life by AI: https://x.com/Rainmaker1973/status/1844755006500573424

* First Day of School: https://www.youtube.com/watch?v=8-4vszvBXIo

* The Simpsons: https://www.youtube.com/watch?v=GUWxoP1pmis

# Others

* AI Storytellers: https://curiousrefuge.com/

# Not categorized yet

* https://hotshot.co/ https://x.com/HotshotSupport

* https://www.tavus.io/ https://x.com/heytavus

Source:  s-marke

https://gist.github.com/s-macke/5d898d529b55d76c531a9b539f2be1e4#file-video-generation-tools-md

 <script src="https://gist.github.com/s-macke/5d898d529b55d76c531a9b539f2be1e4.js"></script>

Realistic Image Generation using AI

 From my tests as shown in the screenshots below, the more realistic images generated by AI are all dependent on how detailed and clear the prompts supplied are, including the negative prompts [what to eliminate] as well as the AI program used - its models and the processing capability of your computer - especially your computer GPU.

The AI software l am currently using is Fooocus 2.5.5 in realistic mode. My primary workstation which houses an AMD Ryzen 9 3900X 12-Core processor, 64Gb RAM and an Nvidia RTX 3090 with 23Gb VRAM. The CPU usage doesn't top 12%, the RAM usage doesn't exceed 22Gb in my tests but the GPU usage tops around 91%

AI Character Image Generator using Machine Learning [ML]

AI Character Image Generator Links on Fooocus

Fooocus: Focus on prompting and generating. 

Fooocus is an image generating software (based on Gradio ).

"Fooocus presents a rethinking of image generator designs. The software is offline, open source, and free, while at the same time, similar to many online image generators like Midjourney, the manual tweaking is not needed, and users only need to focus on the prompts and images. Fooocus has also simplified the installation: between pressing "download" and generating the first image, the number of needed mouse clicks is strictly limited to less than 3. Minimal GPU memory requirement is 4GB (Nvidia)."

 

Github hosted AI character generator forked links include:

• mbah360/Fooocus: Focus on prompting and generating
• mbah360/stable-diffusion-webui-forge
• mbah360/ComfyUI: The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
• mbah360/gradio: Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
• 
  
• Gradio - For making demos of ML models with a web interface - https://www.gradio.app/
• FastRTC - The Real-Time Communication Library for Python - https://fastrtc.org/

The core linux source file is about 10.3Mb zipped file. Unzipped its a 15.5Mb folder with files. The windows zipped file is 1.9Gb which extracts to 5.4Gb. Please note you are recommended to have a free space of at least 40Gb

I personally use the Nvidia Studio driver (professional apps) always  - not the game ready driver.

Note the system requirements, for example for a Windows/Linux system using an Nvidia 4xxx system, its recommended you have a GPU RAM of 4Gb, and a system ram of 8Gb. My current Lenovo 5i laptop supports/exceeds this: RTX4060 with 8Gb VRAM and system RAM 32Gb.  

ML Generation Playlist - Click below for the videos:

ML AI Playlist

Below are sample images generated:

Side notes:

Related youtube shorts on this:  https://www.youtube.com/shorts/U2G-izpTUD8

ML (Machine Learning) Environment setup notes

 1)

Python version check

2)

Nvidia Cuda version check

3)

Pytorch installation using pip

Reference:  https://pytorch.org/get-started/locally/

Test:

GTC March 2025 Keynote with NVIDIA CEO Jensen Huang

March 18th 2025:

Blender Benchmark Results

 Results using Blender 3D software. https://www.blender.org/

1)

Running on my Lenovo laptop

2)