Creating your own LORAs

I’ve been gaining experience with PonyDiffusion and enjoying the easy knowledge it has of MLP characters, but run into some problems with fringe characters it has misconceptions of (characters are the wrong gender, have 90% correct features but random horns or other features, presented in biases acitons or poses, etc). I sometimes find pre-existing LORA’s that help with this, but sometiems the LORA’s themselves cause similar hiccups or interact with existing projects in a negative way.

I’ve recently started dabbling with FLUX2, and while it seems much more useful in creative direction and revision, it really seems to have no concepts of the pony world. Which gets me considering getting into LORA creation. There must be some potential here, seeing how impressive the PonyDiffusion project’s results are for the fandom as a whole and how little the base models seem to know.

I’m under the impression that making a LORA is simply a matter of organizing samples, and relativly trivial to run if you are willing to use a cloud serivce?

I’ve seen some others on here referencing their own LORA creation, and wondering what your experiences have been like. Is this an unrealistic thing to consider doing solo for specific characters, or do any of you find it useful? Is it relatively simple on a base model like SDXL/PXL, and unrealistic on a pony-naive model like Flux?

I think I’m qualified to comment on this subject as I’ve made over 200 MLP character LoRAs for SDXL-based models (mostly Illustrious, though I started with PDXL 6 and have recently shifted to NoobAI).

One thing I have found is that a small, varied and well-tagged dataset is far better than a large one with just passable tagging and not that much variety. I generally use around 30 distinct images of the character (a minimum of around a dozen pics and a maximum of around 50) when training my LoRAs.

After I gather the images for the dataset, I run them all through an autotagger and then touch up its results (adding a tag for the character, removing tags that don’t apply, adding missing ones).

I think I have the capacity to train LoRAs for Flux.2, but I have no clue how to tag images for one (at the moment), so I haven’t done anything of that sort yet.

Thanks for sharing! That seems more approachable than I thought. Do you find you need to do anything special in order to end up using many of your LORAs at the same time without them interacting weirdly?

@Equum
Sadly, I do not know of any ways of getting two different character LoRAs to work together other than decreasing both their strengths and hoping for the best or resorting to inpainting.