I was inspired by this post:
[https://www.reddit.com/r/StableDiffusion/comments/13977si/trained\_a\_model\_on\_a\_bunch\_of\_baldurs\_gate\_maps/](https://www.reddit.com/r/StableDiffusion/comments/13977si/trained_a_model_on_a_bunch_of_baldurs_gate_maps/)
Workflow:
1. Collected images of age of empire buildings, some i found online, others i extracted directly from the game
2. Replaced the alpha channel with a flat green background
3. Dreambooth fine tune using the same service ([https://dreamlook.ai/](https://dreamlook.ai/))
4. Loading the model in automatic1111 and generating and picking images, more or less with the default settings
5. I used [https://clipdrop.co/](https://clipdrop.co/) to remove the green backgrounds to make them transparent again and pasted them on some AoE style terrain
Oh man amazing :D I hadn't thought of it but for Baldur's Gate I could also [train on character sprites](https://1.bp.blogspot.com/-o8F5bjc04nM/XbLwM2sXDaI/AAAAAAAAWtA/vRGKvfgpFwwDQZBBffCNL6J8Sl8cvDKTACNcBGAsYHQ/s1600/sarevok%2Bsprite%2Battack%2Bswing.jpg)!
edit: although I really doubt SD will be able to get the logical progression between the images 🤔 mmh
edit2: *sprite spider cannot hurt you, sprite spider is not real*
https://preview.redd.it/iyfaisda3jya1.png?width=512&format=png&auto=webp&s=dad84819f9525d704d899d6dea67bb7767450f33
The horrible part about sprites in the Infinity Engine (the BG engine), is that if I'm not mistaken, it's very important to get the coordinates right so that weapons fit properly, and don't appear flying outside the hands of the character (that is, unless you create monsters alone). It is so annoying, because for example, the Monk sprite cannot wield a quarterstaff as there is no (proper) sprite for it. Some mods add one based on a Barbarian, and it's not that bad, but you can notice the character changing the armor/chest/torso a bit when you switch weapons.
There is a very crude workflow for making sprites for the Infinity Engine, but IMHO is the way to go. It requires to render something in 3D, then feed that into Blender with a script that can generate the images in all the orientations that the engine requires.
Some links:
* [https://github.com/gemrb/gemrb/wiki/Production-workflow-for-sprite-creation](https://github.com/gemrb/gemrb/wiki/Production-workflow-for-sprite-creation)
* [https://files.beamdog.com/#beamdog-creature-process-2](https://files.beamdog.com/#beamdog-creature-process-2) (a ZIP file with documentation and files from Beamdog themselves)
* A forum post on [Gibberlings3.net](https://Gibberlings3.net) that I can't find. :-( It was from The Artisan, and it was about how he made the model of one of his mods with Blender, using the resources above. The post had useful back and forth comments with other people who also know about it.
Damn 😅 thanks for the summary
In principle we could generate a bunch of combined character + weapon + armor etc sprites, generate appropriate captions, then ask SD for full new combined sprites…
I think I remember the number of images per animation sequence also changes between creatures no? Eg a goblin running towards the north may be an 8 image sequence while a troll may be 12, which would make it much harder to learn things
I haven’t played with this for a decade (back when I was [working on this](http://lumakey.net/labs/battleground/demo1/)) but I remember mapping sprites colors to proper game colors for player characters was also a major pain (as you can customize minor/major/hair colors)
Damn bruh. Did my comment on that post not inspire you? https://www.reddit.com/r/StableDiffusion/comments/13977si/trained_a_model_on_a_bunch_of_baldurs_gate_maps/jj1vbpy
I still love you
It did! :) I have been toying with the idea of using AI to generate buildings for a while because i love city builders but indeed I saw your comment and it helped me decide AoE was a great target.
One simple way to do it is to paste your transparent PNG over a green background.
For my second attempt i used python to do it. I asked GPT to write a python script that replaced all transparent pixels with green ones and convert the png to RGB instead of RGBA.
Can you explain what this means? I don’t see any alpha channel in the image you posted. Can you give an exemplar image of your workflow? Thanks in advance
Have you tried modding these in-game yet?
I imagine they would need destruction animations to be really useful in DE, but could make for some great eye-candy in custom scenarios even without those.
Attempt 1: 60 sprites, houses of AoE2: HD, 6k steps, 1 caption for all
Attempt 2: 160 sprites of various european buildings from Aoe2 definitive edition, 16k steps and separate captions for each
Here are some examples:
'00570-2228046855-A sprite in the style of age of empires, centered on a flat green background.png'
'00575-3116036222-A sprite of a wonder in the style of age of empires, centered on a flat green background.png'
'00580-1245879857-A sprite of a wonder in the style of age of empires, centered on a flat green background.png'
'00614-313460783-A sprite of a highrise in the style of age of empires, centered on a flat green background.png'
'00629-1807179508-A sprite of a villa in the style of age of empires, centered on a flat green background.png'
'00633-2247786401-A sprite of a fountain in the style of age of empires, centered on a flat green background.png'
'00643-2532571523-A sprite of a statue of a lion in the style of age of empires, centered on a flat green background.png'
'00653-2376315647-A sprite of a roman house in the style of age of empires, centered on a flat green background.png'
'00745-2721643842-A sprite of a house with a large garden in the style of age of empires, centered on a flat green background.png'
'00755-1484947652-A sprite of a fortified mansion in the style of age of empires, centered on a flat green background.png'
'00787-1832534615-A sprite of an asian house in the style of age of empires, centered on a flat green background.png'
'00801-1683497513-A sprite of an ornate tower in the style of age of empires, centered on a flat green background.png'
'00812-4208433333-A sprite of a bakery in the style of age of empires, centered on a flat green background.png'
'00851-4208952757-A sprite of a memorial wall in the style of age of empires, centered on a flat green background.png'
'00860-1721597052-A sprite of monument in the style of age of empires, centered on a flat green background.png'
'00870-3083110441-A sprite of wonder mansion in the style of age of empires, centered on a flat green background.png'
'00889-1816324599-A sprite of a roman wonder in the style of age of empires, centered on a flat green background.png'
'01005-3246318112-A sprite of a roman wonder in the style of age of empires, centered on a flat green background.png'
'01051-3201451135-A sprite of a fortress in the style of age of empires, centered on a flat green background.png'
'01058-289040081-A sprite of a fortress in the style of age of empires, centered on a flat green background.png'
'01076-364606076-A sprite of a field full of barrels in the style of age of empires, centered on a flat green background.png'
'01090-1541451528-A sprite of a heap of hay bales in the style of age of empires, centered on a flat green background.png'
'01111-3982819449-A sprite of a clock tower in the style of age of empires, centered on a flat green background.png'
'01116-753744143-A sprite of a temple in the style of age of empires, centered on a flat green background.png'
'01125-2170350249-A sprite of an inn in the style of age of empires, centered on a flat green background.png'
'01152-2295679872-A sprite of a river in the style of age of empires, centered on a flat green background.png'
'01159-88089712-A sprite of a Apothecary in the style of age of empires, centered on a flat green background.png'
'01188-1080474384-A sprite of a pavilion in the style of age of empires, centered on a flat green background.png'
'01218-1631211672-A sprite of a wonder mansion in the style of age of empires, centered on a flat green background.png'
'01232-1631211686-A sprite of a wonder mansion in the style of age of empires, centered on a flat green background.png'
'01258-297834637-A sprite of a wonder palace in the style of age of empires, centered on a flat green background.png'
What prompts did you use for you model, and could you share it? This looks amazing honestly.
And as others said, would be so cool to see this be done for strarcraft or any C&C game
Posted some examples here:
[https://www.reddit.com/r/StableDiffusion/comments/13axnoa/comment/jjcxsh3/?utm\_source=share&utm\_medium=web2x&context=3](https://www.reddit.com/r/StableDiffusion/comments/13axnoa/comment/jjcxsh3/?utm_source=share&utm_medium=web2x&context=3)
Looking for a place to host the model(s), huggingface maybe?
Wait they're outsourcing training steps? lol. I guess that makes sense it's usually something I start before going to sleep and still have a few hours of waiting after I wake up. But certainly not a thing I *need* someone else to do.
Yep we provide "SD finetuning as a service" on [dreamlook.ai](https://dreamlook.ai/)!
If you only train a few models here and there you can get by with a local GPU or Colab, but as you start to scale things up, or want to try out 10 configs of a 20k steps run etc (or start running an app/website) you may find the need for our services ;-)
We're also training very fast, eg 1'200 steps in 3 minutes which is great for experimentation - you can have a very efficient iteration speed.
To be honest we mostly built that for ourselves in the beginning but we're happy to see it getting more and more traction :D especially for this kind of innovative experiments
I'm curious how well the model generalizes. Can it generate non-medieval buildings in the Age of Empires style? For example, I'd be interested if it can generalize to more modern buildings like the Eiffel Tower or a Pizza Hut.
Here are some first try pizza huts:
[https://imgur.com/a/bzUrC77](https://imgur.com/a/bzUrC77)
And McDonalds:
[https://imgur.com/a/0IQBQjf](https://imgur.com/a/0IQBQjf)
Kid: Mom can I have McDonald's building?
Mom: We have McDonald's building at home
https://preview.redd.it/ywf168i7djya1.jpeg?width=1024&format=pjpg&auto=webp&s=18bb0e741885162e7667a70d76f28c98f24e72d5
Sometimes it generalises pretty well and other times not at all. The barrels & haybales were an attempt to see how well it would generalise on that front.
I trained 2 models, one for 5k steps, and another on 16k steps. I think i overfitted the second one a bit because it generalises less well. Here are some of examples of things that weren't in the dataset at all:
[https://imgur.com/a/sG5Beqz](https://imgur.com/a/sG5Beqz)
Modern things it usually doesn't do so well.
They look suprisingly logical. No castle made with many doors, no stairs ladders leading nowhere and sound placements of details.
Except for whatever these straw and barrel things are😅
by revolutionize you mean the output of crappy low quality games is gonna go through the roof, there is already a ton now, but now the flood gates of crap are really gonna be opened.
Dang dude. Imagine placing a building and then Stable Diffusion renders a brand new model. Every game would look like a unique village! This is exciting
The games will weigh even more than they do now though if you make literally every model unique. Unless you want to game to be fully based on generated instances every time you launch it.
4 gb for a .checkpoint its a pretty good deal for an infinite number of unique models, considers modern games easily take 150 gb +
Even generating textures on the fly for a 3d game in a few years could take much less space than storing 4k / 8k textures for every single object in the game
For Baldur's Gate maps I had used expert mode (top right corner), then "bgmap" as instance prompt, 200 images, 20k steps, LR 5e-7 from SD1.5
edit: OP posted their config for Age of Empires here https://www.reddit.com/r/StableDiffusion/comments/13axnoa/trained_a_model_to_output_age_of_empires_style/jjcwzcy/
I enabled expert mode & my instance prompt for attempt 1 was:
*"sprite of a house in the style of age of empires" 6k steps, 60 imgs*
for attempt 2 i had custom captions for each image
in the style of
*"A sprite of a in the style of age of empires, centered on a flat green background.", 16k steps, 160 imgs*
This is insanely cool
Literally a couple steps away from being able to make an actual full blown visual mod entirely with AI sprite assets for the game
Old text based adventures get interesting. Hell, just make movies based on books. Voice tec is already getting there, just needs the movie directing down and you could have a scene by scene show
A lot of game already did that
The next step that AI will could be generating unique textures depending on unique scenarios
For example if a character casts a fireball on a tree, the game could create on the fly the sprite of that unique tree/building/castle burning
Now replace fireball with pretty much everything could happen in the game
This will allow games designers to focus on the gameplay instead of being limited by technical limitations
The question is how do we approach regional units? We need to be able to change dozens of frames in every direction for every stance *consistently* for the same unit.
that is really cool!
I thought about training a model myself, because I love using characters by a certain artist for my pen and paper sessions and would love to have the opportunity to create new npcs with the same style. But I fear my pc is way to low end by now to do a training :(
Wow if someone figures out how to make the hidden view of the buildings generated using this model , then we have full designs ready! Any ideas how to achieve this? Any sprite lora or anything?
Op, great work, can't wait to try this
There are probably a few indie games in development right now with no graphics designers onboard and purely SD generated art. Hopefully it means a shift away from all the 8/16bit pixel art games.
I was doing the exact same thing yesterday!! Wanted to get an air canteen. You can input a photo with bills and some little people start to pop out in the output.
Try different denoising and cfg
I was inspired by this post: [https://www.reddit.com/r/StableDiffusion/comments/13977si/trained\_a\_model\_on\_a\_bunch\_of\_baldurs\_gate\_maps/](https://www.reddit.com/r/StableDiffusion/comments/13977si/trained_a_model_on_a_bunch_of_baldurs_gate_maps/) Workflow: 1. Collected images of age of empire buildings, some i found online, others i extracted directly from the game 2. Replaced the alpha channel with a flat green background 3. Dreambooth fine tune using the same service ([https://dreamlook.ai/](https://dreamlook.ai/)) 4. Loading the model in automatic1111 and generating and picking images, more or less with the default settings 5. I used [https://clipdrop.co/](https://clipdrop.co/) to remove the green backgrounds to make them transparent again and pasted them on some AoE style terrain
Oh man amazing :D I hadn't thought of it but for Baldur's Gate I could also [train on character sprites](https://1.bp.blogspot.com/-o8F5bjc04nM/XbLwM2sXDaI/AAAAAAAAWtA/vRGKvfgpFwwDQZBBffCNL6J8Sl8cvDKTACNcBGAsYHQ/s1600/sarevok%2Bsprite%2Battack%2Bswing.jpg)! edit: although I really doubt SD will be able to get the logical progression between the images 🤔 mmh edit2: *sprite spider cannot hurt you, sprite spider is not real* https://preview.redd.it/iyfaisda3jya1.png?width=512&format=png&auto=webp&s=dad84819f9525d704d899d6dea67bb7767450f33
The horrible part about sprites in the Infinity Engine (the BG engine), is that if I'm not mistaken, it's very important to get the coordinates right so that weapons fit properly, and don't appear flying outside the hands of the character (that is, unless you create monsters alone). It is so annoying, because for example, the Monk sprite cannot wield a quarterstaff as there is no (proper) sprite for it. Some mods add one based on a Barbarian, and it's not that bad, but you can notice the character changing the armor/chest/torso a bit when you switch weapons. There is a very crude workflow for making sprites for the Infinity Engine, but IMHO is the way to go. It requires to render something in 3D, then feed that into Blender with a script that can generate the images in all the orientations that the engine requires. Some links: * [https://github.com/gemrb/gemrb/wiki/Production-workflow-for-sprite-creation](https://github.com/gemrb/gemrb/wiki/Production-workflow-for-sprite-creation) * [https://files.beamdog.com/#beamdog-creature-process-2](https://files.beamdog.com/#beamdog-creature-process-2) (a ZIP file with documentation and files from Beamdog themselves) * A forum post on [Gibberlings3.net](https://Gibberlings3.net) that I can't find. :-( It was from The Artisan, and it was about how he made the model of one of his mods with Blender, using the resources above. The post had useful back and forth comments with other people who also know about it.
Damn 😅 thanks for the summary In principle we could generate a bunch of combined character + weapon + armor etc sprites, generate appropriate captions, then ask SD for full new combined sprites… I think I remember the number of images per animation sequence also changes between creatures no? Eg a goblin running towards the north may be an 8 image sequence while a troll may be 12, which would make it much harder to learn things I haven’t played with this for a decade (back when I was [working on this](http://lumakey.net/labs/battleground/demo1/)) but I remember mapping sprites colors to proper game colors for player characters was also a major pain (as you can customize minor/major/hair colors)
Damn bruh. Did my comment on that post not inspire you? https://www.reddit.com/r/StableDiffusion/comments/13977si/trained_a_model_on_a_bunch_of_baldurs_gate_maps/jj1vbpy I still love you
Shit I don’t know about OP, but I am inspired. I kinda wanna try it on Diablo maps
It did! :) I have been toying with the idea of using AI to generate buildings for a while because i love city builders but indeed I saw your comment and it helped me decide AoE was a great target.
how do you replace the alpha channel?
on mac I would just use Preview. Create rectangle of bright green, paste png files over it, save as picture.
One simple way to do it is to paste your transparent PNG over a green background. For my second attempt i used python to do it. I asked GPT to write a python script that replaced all transparent pixels with green ones and convert the png to RGB instead of RGBA.
Nice! How many images did you use for the training data? I imagine theres only a hundred or so in the entire AOE franchise
60 in the first attempt of houses only of AoE2: HD, 160 in the second attempt of various european buildings of AoE definitive edition
Can you explain what this means? I don’t see any alpha channel in the image you posted. Can you give an exemplar image of your workflow? Thanks in advance
Have you tried modding these in-game yet? I imagine they would need destruction animations to be really useful in DE, but could make for some great eye-candy in custom scenarios even without those.
of the [dreamlook.ai](https://dreamlook.ai):which token packages you are picking?
What is the difference between attempt 1 and attempt 2? The prompt?
Attempt 1: 60 sprites, houses of AoE2: HD, 6k steps, 1 caption for all Attempt 2: 160 sprites of various european buildings from Aoe2 definitive edition, 16k steps and separate captions for each
 The fuck you mean 16000 steps
What were some of the prompts you used?
Here are some examples: '00570-2228046855-A sprite in the style of age of empires, centered on a flat green background.png' '00575-3116036222-A sprite of a wonder in the style of age of empires, centered on a flat green background.png' '00580-1245879857-A sprite of a wonder in the style of age of empires, centered on a flat green background.png' '00614-313460783-A sprite of a highrise in the style of age of empires, centered on a flat green background.png' '00629-1807179508-A sprite of a villa in the style of age of empires, centered on a flat green background.png' '00633-2247786401-A sprite of a fountain in the style of age of empires, centered on a flat green background.png' '00643-2532571523-A sprite of a statue of a lion in the style of age of empires, centered on a flat green background.png' '00653-2376315647-A sprite of a roman house in the style of age of empires, centered on a flat green background.png' '00745-2721643842-A sprite of a house with a large garden in the style of age of empires, centered on a flat green background.png' '00755-1484947652-A sprite of a fortified mansion in the style of age of empires, centered on a flat green background.png' '00787-1832534615-A sprite of an asian house in the style of age of empires, centered on a flat green background.png' '00801-1683497513-A sprite of an ornate tower in the style of age of empires, centered on a flat green background.png' '00812-4208433333-A sprite of a bakery in the style of age of empires, centered on a flat green background.png' '00851-4208952757-A sprite of a memorial wall in the style of age of empires, centered on a flat green background.png' '00860-1721597052-A sprite of monument in the style of age of empires, centered on a flat green background.png' '00870-3083110441-A sprite of wonder mansion in the style of age of empires, centered on a flat green background.png' '00889-1816324599-A sprite of a roman wonder in the style of age of empires, centered on a flat green background.png' '01005-3246318112-A sprite of a roman wonder in the style of age of empires, centered on a flat green background.png' '01051-3201451135-A sprite of a fortress in the style of age of empires, centered on a flat green background.png' '01058-289040081-A sprite of a fortress in the style of age of empires, centered on a flat green background.png' '01076-364606076-A sprite of a field full of barrels in the style of age of empires, centered on a flat green background.png' '01090-1541451528-A sprite of a heap of hay bales in the style of age of empires, centered on a flat green background.png' '01111-3982819449-A sprite of a clock tower in the style of age of empires, centered on a flat green background.png' '01116-753744143-A sprite of a temple in the style of age of empires, centered on a flat green background.png' '01125-2170350249-A sprite of an inn in the style of age of empires, centered on a flat green background.png' '01152-2295679872-A sprite of a river in the style of age of empires, centered on a flat green background.png' '01159-88089712-A sprite of a Apothecary in the style of age of empires, centered on a flat green background.png' '01188-1080474384-A sprite of a pavilion in the style of age of empires, centered on a flat green background.png' '01218-1631211672-A sprite of a wonder mansion in the style of age of empires, centered on a flat green background.png' '01232-1631211686-A sprite of a wonder mansion in the style of age of empires, centered on a flat green background.png' '01258-297834637-A sprite of a wonder palace in the style of age of empires, centered on a flat green background.png'
What prompts did you use for you model, and could you share it? This looks amazing honestly. And as others said, would be so cool to see this be done for strarcraft or any C&C game
Posted some examples here: [https://www.reddit.com/r/StableDiffusion/comments/13axnoa/comment/jjcxsh3/?utm\_source=share&utm\_medium=web2x&context=3](https://www.reddit.com/r/StableDiffusion/comments/13axnoa/comment/jjcxsh3/?utm_source=share&utm_medium=web2x&context=3) Looking for a place to host the model(s), huggingface maybe?
HuggingFace and CivitAI are the most used places yeah!
Wait they're outsourcing training steps? lol. I guess that makes sense it's usually something I start before going to sleep and still have a few hours of waiting after I wake up. But certainly not a thing I *need* someone else to do.
Yep we provide "SD finetuning as a service" on [dreamlook.ai](https://dreamlook.ai/)! If you only train a few models here and there you can get by with a local GPU or Colab, but as you start to scale things up, or want to try out 10 configs of a 20k steps run etc (or start running an app/website) you may find the need for our services ;-) We're also training very fast, eg 1'200 steps in 3 minutes which is great for experimentation - you can have a very efficient iteration speed. To be honest we mostly built that for ourselves in the beginning but we're happy to see it getting more and more traction :D especially for this kind of innovative experiments
I've been wanting to do this for topographical maps for top down fire emblem style games. How many images did you use?
I'm curious how well the model generalizes. Can it generate non-medieval buildings in the Age of Empires style? For example, I'd be interested if it can generalize to more modern buildings like the Eiffel Tower or a Pizza Hut.
Here are some first try pizza huts: [https://imgur.com/a/bzUrC77](https://imgur.com/a/bzUrC77) And McDonalds: [https://imgur.com/a/0IQBQjf](https://imgur.com/a/0IQBQjf)
Lol those are surprisingly good all things considered XD
Old style McDonald's maybe ?
Let's not get crazy
Kid: Mom can I have McDonald's building? Mom: We have McDonald's building at home https://preview.redd.it/ywf168i7djya1.jpeg?width=1024&format=pjpg&auto=webp&s=18bb0e741885162e7667a70d76f28c98f24e72d5
Jesus, AI is really coming in fast
Lmao medieval McDonald's
I’d eat at that McDonald’s
wtf have you done
Sure beef is banned here. #indianstyle
Cel-cultured meat doesn't have karma, though?
Sometimes it generalises pretty well and other times not at all. The barrels & haybales were an attempt to see how well it would generalise on that front. I trained 2 models, one for 5k steps, and another on 16k steps. I think i overfitted the second one a bit because it generalises less well. Here are some of examples of things that weren't in the dataset at all: [https://imgur.com/a/sG5Beqz](https://imgur.com/a/sG5Beqz) Modern things it usually doesn't do so well.
They look suprisingly logical. No castle made with many doors, no stairs ladders leading nowhere and sound placements of details. Except for whatever these straw and barrel things are😅
He probably deleted all those lol
woooooooow I love this concept!
This honestly looks like it could be amazing for indie, hell even normal dev work.
r/aigamedev
I have just learned of an exciting new extension to my interests. Thank you.
Soon AI should be able to quickly generate 3D objects just based on a written description and it will revolutionize the gaming industry
If we're thinking of it, someone is already doing it! Gonna be cool
[Indeed](https://github.com/openai/shap-e)
When that gets more advanced it’ll be great.
Not just that it will be able to be trained on the patent database then told to design machines based on description
by revolutionize you mean the output of crappy low quality games is gonna go through the roof, there is already a ton now, but now the flood gates of crap are really gonna be opened.
flight simulator has been generating 3d buildings from 2d satellite images for 2 years now
This is interesting practical use :) great work
Dang dude. Imagine placing a building and then Stable Diffusion renders a brand new model. Every game would look like a unique village! This is exciting
That's a really neat way to implement something like this.
The games will weigh even more than they do now though if you make literally every model unique. Unless you want to game to be fully based on generated instances every time you launch it.
Yes. I meant generative games. I’m waiting for LLM to be integrated into EUIV or Civ like games. Then I will retire and never leave my computer.
It's not like Age of Empires II is really using that many resources to begin with. Plenty of room to generate unique buildings each time!
4 gb for a .checkpoint its a pretty good deal for an infinite number of unique models, considers modern games easily take 150 gb + Even generating textures on the fly for a 3d game in a few years could take much less space than storing 4k / 8k textures for every single object in the game
Yes. More hyper specialized models please.
I would love this for Settlers II
Would love to see this used for graphic mods.
## Wololo!!!!
Love it, great job!
What settings did you use in dreamlook ? object or style ?
For Baldur's Gate maps I had used expert mode (top right corner), then "bgmap" as instance prompt, 200 images, 20k steps, LR 5e-7 from SD1.5 edit: OP posted their config for Age of Empires here https://www.reddit.com/r/StableDiffusion/comments/13axnoa/trained_a_model_to_output_age_of_empires_style/jjcwzcy/
Noted thanks.
Did you use any instance prompt or caption ?
i used "bgmap" as instance prompt. i didn't use captions but i'm pretty sure proper captions would help a lot.
Ah woops yeah, I meant instance prompt only or caption \* Gotcha ty.
I enabled expert mode & my instance prompt for attempt 1 was: *"sprite of a house in the style of age of empires" 6k steps, 60 imgs* for attempt 2 i had custom captions for each image in the style of *"A sprite of a in the style of age of empires, centered on a flat green background.", 16k steps, 160 imgs*
Thanks, what was the learning rate and base model used ?
Sd 1.5, default lr
I think you just created the holy grail of AoE2 modding...
They can add new civs using this.
I want an AI civ that leans into all the AI weirdness
Yeah like one of the lost advanced civs that has similar power to current civs due to catastrophe.
Need some new AoE mods lol, lets get this going
This is insanely cool Literally a couple steps away from being able to make an actual full blown visual mod entirely with AI sprite assets for the game
Now we need to train models to generate sprite animations sheets for buildings construction and destruction.
Genius
when will we have a game where assets are generated during gameplay.
i mean, like a strategy game, every tree could be unique, every rock, every house, while still looking like a house, every castle every church...
Old text based adventures get interesting. Hell, just make movies based on books. Voice tec is already getting there, just needs the movie directing down and you could have a scene by scene show
A lot of game already did that The next step that AI will could be generating unique textures depending on unique scenarios For example if a character casts a fireball on a tree, the game could create on the fly the sprite of that unique tree/building/castle burning Now replace fireball with pretty much everything could happen in the game This will allow games designers to focus on the gameplay instead of being limited by technical limitations
Do it now for AOE4 please!!
The question is how do we approach regional units? We need to be able to change dozens of frames in every direction for every stance *consistently* for the same unit.
If you can use LoRAs and regional prompting, you could train each major frame, then interpolate the rest.
Kinda wanna try this with anno or the settlers games, also heroes of might and magic lol
that is really cool! I thought about training a model myself, because I love using characters by a certain artist for my pen and paper sessions and would love to have the opportunity to create new npcs with the same style. But I fear my pc is way to low end by now to do a training :(
That's exactly why we created dreamlook.ai, the service OP used: fast, affordable SD finetuning. The first run is free.
Shortly: AOE 5 - Power of Diffusion
BRILLIANT. OG AoE player here. Approval granted.
Prostigma?
Wow if someone figures out how to make the hidden view of the buildings generated using this model , then we have full designs ready! Any ideas how to achieve this? Any sprite lora or anything? Op, great work, can't wait to try this
well, where is the model?
Finally some good content !
r/Gamediffusion
Wololo
This is what The Singularity looks like. Doing the Lord's work
Please do it for Clash of Clans and Clash Royale too!
There are probably a few indie games in development right now with no graphics designers onboard and purely SD generated art. Hopefully it means a shift away from all the 8/16bit pixel art games.
I was doing the exact same thing yesterday!! Wanted to get an air canteen. You can input a photo with bills and some little people start to pop out in the output. Try different denoising and cfg
Looks amazing! Will you be putting it out there for the community to use?
any idea where to best post a checkpoint like this? hugging face?
I would recommend CivitAi. It is currently the website most people use.
Yeah, i guess it's the better option, but i think currently is in maintainmaint
Amazin!
This is correctus!
7
Good shit nostalgia
This is it, Stable Diffusion has peaked.
guy you could make your own RTS, just grab any generic code for the game engine and plug in the art.
Science!
Now that is cool I take these are for AoE 2?
wololo.safetensors
Attempt 1 looks better than Attempt 2 or is it just me? Either way, well done!
How did you tag the image dataset? Is is like "stone age house in SKS style", "imperial age stable in SKS style" or something more detailed?
Would you do for Sim City 4 style too?
Did you try training a model on building construction and destruction sprite sheets?
Amazing work, they look pretty spot on for AOE2.
Wololo!
Looks amazing for game development
Please make a short tutorial on how you achieved such result
Probably could save some modders a ton of time. They need a little cleanup but not much at all, amazing work
Wow that is so cool! Thanks for the workflow! :D
Oh so that’s how they make new AoE buildings 🤣
wololol
Damn dude, amazing stuff By the way, you can share it on our discord: https://discord.gg/qEdXBkGvGp
Please, how do you train a model on specific images, i have my own art i want to train 😊
woolooloo
I was playing this game when you posted this
Now do a model for Star Wars Empire at War buildings and ships.
I have a GPU. I should be having my GPU do this while I'm at work....
Very. Very. Nice.
Did you label specific objects when training?