€10

C4D OpenPose Rig for Stable Diffusion [3D C4D → AI Stable Diffusion]

I want this!

C4D OpenPose Rig for Stable Diffusion [3D C4D → AI Stable Diffusion]

€10

OpenPose Custom Character Rig in Cinema 4D scene (3D) using any kind of rigged character animation (Rokoko, Mixamo and more...), use it to create AI generated images/videos (with Controlnet in Stable Diffusion).

This is a tool I've been using in personal/professional productions. It allows me to get the most precise control over character animation and directing over AI generations. I feel that this workflow/setup allows a full creative control over AI. Can't wait to see what people will be able to do with this :)

WORKFLOW

  • Stable Diffusion (AI / Automatic1111) → txt2img / Deforum, Frame Interpolation, Upscale
  • C4D (3D / v.2024.4.1) → Custom Open Pose Rig + Viewport renderer
  • Octane Material (3D) - Legacy friendly
  • Rokoko Studio (Motion capture / release-2.4.4.7 ) → motion capture suit
  • After effects / Media Encoder (Montage, Editing)

WHAT'S IN THE PACKAGE

  • 1 C4D Scene file
  • 1 Documentation file
  • Stable diffusion image generation .txt file

SUMMARY

This project is based on the retranscription of a color code to render a character animation using 3D software (here C4D) that is ready to use in Stable Diffusion. The name of resource is "OpenPose COCO 18 Color Reference Chart" by U/GBJI (cf. spreadsheet).

/!\ DO NOT CHANGE THE COLOR OF THE LAYER AND MATERIALS / THEY ALL RESPOND TO A #HEX CODE. IF YOU DO THE OPENPOSE RIG WON'T WORK PROPERLY. /!\


OpenPose Bone color scheme → Cinema4D (2024.4.1) custom rig setup. Update: the C4D interface screenshot is an old C4D file with wrong colors that has been corrected :)

HOW TO MAKE IT WORK ?

SETUP C4D FILE:

  • Be sure that the .C4D file has those "Color Management'' Parameters. (In "Edit" → "Setting Scene" or keyboard shortcuts to get this "Project" window is: Ctrl + D).
  • Also, you NEED to have your viewport shading mode in Constant Shading "Display" → select "Constant Shading".

IMPORT CHARACTER ANIMATION SOURCE + SETUP THE OPEN POSE RIG:

  • In the scene, import the character animation you would like to transfer to the OpenPose Rig (it can be from Mixamo, Rokoko...etc).
  • The OpenPose Rig is composed of Spheres (Joints) and Sweep objects (Pairs Bones) that you can find in the "Objects tab", under the null named "Null:OpenPose_Rig".
  • Select the Sphere Objects (named from "0" to "17") one by one and click on their Constraint tag. Now, in the "Attribute tab" → "Clamp" you can modify the "target" to the relevant bone of the animation rig. (ex: 2 →Right shoulder). You don't need to touch the Sweep objects (Pairs Bones), as they are already set up.
  • Put the distance parameter of the Constraint Tag to "0 cm".
  • From this point, the OpenPose Rig should stick and follow the Animation Rig you've just imported.

UPDATE - OCCLUDER OBJ:

  • I've added a "Occlusion" Extrude Object (from a spline) in the "Null:Animation_Rig" as a test to get better results when the character rotates; it can mess up the AI generation.

RENDER PROCESS (IMAGES AND VIDEOS) + RENDER SETTINGS:

  • Double check that your viewport is in "Constant Shading" mode and the Color Management parameters → cf. first part of the doc "SETUP C4D FILE".
  • Be sure that you have a "Sky" Object with a black material (HEX #000000), it will create the black background that's needed for the AI Generation.
  • "Render settings" → Select "Viewport Renderer".
  • "Viewport Renderer Tab" → Copy the "Basic Properties" and "Filter" parameters as the screenshot.
  • Be sure not to lock any Layers of "Null:OpenPose_Rig" otherwise it will appear gray.

QUICK STABLE DIFFUSION INTRODUCTION:

I've used this setup in Stable Diffusion Automatic 1111 (local), in SDXL and SD1.5 / txt2img and Deforum. Here I won't go into the details step-by-step process but I share screenshots of the way I configure Stable Diffusion to get to this result. Take note of the yellow squares, they are important parameters.

Stable Diffusion

  • Check Yellow
  • Uncheck Red

ControlNet (v1.1.452)

  • OpenPose "prepocessor: none" and "Model: "thibaud_xl_openpose [c7b9cadd]"

In the documentation, I'll add the .txt file of this image, so you'll have the full list of parameters.

SHOWCASE "Shaping AI output with 3D & VFX!" (2024.03)

Here's a research I've worked on with this technique, I added more ControlNet from 3D export passes to improve the result and control over the AI generation:

  • ControlNet: T2I-Adapteur - OpenPose - Depth
  • 3D Pass for VFX: Z-Depth - OpenPose Rig - Material ID - Object ID

Please consider supporting this with a fair price as it takes a long time and effort to build that kind of tool. Don't hesitate to tag me on your own projects and experimentations so I can see how far this tool can be pushed!

Special thanks to Alexandre Rivaux (https://www.instagram.com/arivaux/) for his help in learning how to tame AI and Florian Coppier (https://www.instagram.com/pti_lofi/) for helping me find the right way to link AI and 3D.

If you have any questions, you can contact me by email: theo.rocq@gmail.com

📱 Follow me on instagram : https://www.instagram.com/theorocq/


I want this!
Copy product URL
30-day money back guarantee