Leveraging AI for Science Fiction

At the risk of getting called out for doing anything other than working on my Novel…

I’m currently having Grok Build (Cursor) build stuff for me in Blender, images included. These are only basic renders, which is sufficient to help me visualize the scenes I’m trying to write. These are to a reasonable scale so that the references are useful. As things progress I’m going to add details, textures, and all of the necessary stuff so that I can have Blender produce high quality images and video.

The other viable option for this would be to use Unreal Engine for generating your videos. That’s going to be far less abusive on your computer and much faster than Blender, but there are trade offs. You’ll want to check the specifics of your use case. The licensing for Unreal has had very clear allowances for media creation using the engine, but that may be shifting as a result of AI Agents making it vastly more accessible.

When you get into the technical details this is achievable on pretty lightweight hardware. The system I’m currently using is a Ryzen Gen 2 CPU, 32GB of memory, and a Nvidia 1060 GPU. All of it is quite old and slow by any standard. There are well known tricks to do this kind of work with under powered systems and it only took a small recommendation to convince Grok to use those methods. As an example each type of building is a blender asset. The model for the habitat itself simply stores the positioning / placement of each one. This keeps the files much easier to manage. I have 1 house file and when I update it, they all update. I could get creative and do video game tricks to make all the houses look different, but that’s really not necessary for my purposes.

Turning this into high quality or cinematic images/video is just a matter of textures and render time. That’s pretty far down the road for my goals, I kinda have to write the book at some point. Hardware would be a huge constraint for doing video output locally, but there are a number of reputable services that will render scenes from blender for money.

NOTE: I started maybe 2 hours ago dumping in descriptions and building up to what you see here. In the time it’s taken me to write up this post I’m already looking at better renders. I’m also working my day job right now so the level of effort here is 3-4 sentence prompts on top of scenes from my book as context for the agent to build. 95% doing other work and 5% driving the Agent.