Tips for Coding Scalable Addons
Guest author and product creator Spencer Magnusson discusses best practices and strategies for coding scalable add-ons. This article is based on the Blender Artists thread of the same name.
I have made a couple of add-ons for the Blender Market, particularly the Shot Matcher. I have updated the add-on almost a dozen times, and as I’ve applied coding techniques I learned from college and on my own, I thought I’d share some tips on coding in a maintainable way. Keep in mind, this is not a “how-to” or beginner’s look at Python--just techniques to make coding easier to write in the long run, as well as keeping it structured and optimized.
And any other advanced developers, please share things you have learned as well, either here or on my Blender Artists thread!
Code Design and Structure
Honestly, instead of lecturing you for a page, I’ll point you to this website: Refactoring Guru. It covers refactoring and several design patterns, utilizing various programming techniques to make code easier to manage and maintain. It is better to prevent problems than to repair them, so be sure to plan how to write your code. This will save you time down the road and makes feature updates easier.
Managing Multiple Files (or multiple folders)
If your add-on is small enough, putting all your code into one text file would be appropriate. However, for a medium-sized or bigger add-on, you should separate it into files. If there are lots of files, separate those files into folders!
As an organizational principle, try to have every file and folder only do one thing. For example, instead of one file with multiple operator classes for exporting different formats, make a file for each operator, and put them all in an “export” folder. If needed, put any common dependencies (such as functions) in their own file and import them into the other files.
Even with an organized plugin, (un)registering may seem like a headache. Thankfully, Python can help with that. For multiple files in one folder, you can put your components into a list and have Blender just register the list:
For multiple folders, it is a similar idea. Take a look at Jacques Lucke’s code on importing multiple folders (if you use it, be kind and give him credit). Also make sure to include a “__init__.py” in each folder, even if “__init__.py” is an empty file. Python needs this to know which folders to recognize.
Reusing Functions for Similar Processes
As you expand your add-on, you may notice code reused and duplicated throughout the files. Maybe a few of your operators use the same process (or a similar process with minor differences). Maybe a few panels show information the same way. I noticed it with some UI panels, displaying the same PropertyGroup of settings in the same format. I hated having the same code in two places. My answer? Put it all into a function. If there is anything unique, pass it as a function argument. I did this in my Shot Matcher add-on:
And then my panel classes are just this:
Now I have the same information for multiple panels, without having duplicate code!
Managing Space Complexity
There are many ways algorithms can take up space: loading images or movie clips into memory, storing colors and presets, or caching simulation results. Some calculations require lots of data. To programmers, the amount of data is as important as how the space increases based on input.
For example, a hypothetical render of n objects would require n megabytes of memory, or even 10 * n megabytes. Wouldn’t that be nice? A more spatially complex render would require n * n megabytes. Not bad. But what if it was 2 ^ n megabytes? Twenty objects would require a terabyte of memory!! That is why space complexity is important; the memory used by an algorithm must be scalable.
Granted, this only becomes a worry for huge amounts of data, such as multi-dimensional arrays and media. But other stored or generated data can become unused, unnecessary, or quickly outdated. So keep an eye on variables and storage; make sure they do not grow too big, too fast.
Managing Time Complexity
I can use an example similar to the above for time complexity. It would be terrible if a render engine, given n objects, renders in 2 ^ n seconds. Twenty objects would take twelve days to render! Iterating over lots of data and doing heavy calculations can slow down operators. This can add up, especially for users with low-end laptops. Because Blender usually waits for operators to finish, it is important for processes to run efficiently.
One performance issue with using lots of data is searching it. To increase search speed, I have found Blender’s collection properties useful. As long as you know an element’s key, searching is virtually instantaneous. The only requirement is that each key is unique. Some data in Blender is forced to be unique, such as image or movie clip names--use this to your advantage.
Keep the user experience in mind. One expensive process I’ve coded in my Shot Matcher add-on is the video analysis, which reads multiple frames’ pixels. I knew that analyzing every frame was unnecessary and could take several minutes to process. So I gave the user control of that level of complexity by presenting input for the range of frames to iterate over. That way, they choose the number of frames to calculate. In short, a simple way to manage time complexity is to leave it up to the user (but be sure to give helpful default values to guide their decision).
Another method worth considering is to exchange accuracy for speed. One component of Blender does this very well: Eevee. We think accuracy is always essential, but users only need so much (and this varies from user to user). If a somewhat inaccurate algorithm satisfies the user’s needs, it may be worth leaving alone.
If the process takes a long time even for fast computers, consider designing it to be handled on multiple threads or processes. Python has plenty of documentation on how to do this; be sure to visit Blender’s notes on threading and make sure you clean up threads and processes properly.
People get in strangely heated debates as to whether code should be commented--from “everything should be commented” to “it needs to be cast out like the default cube”. Regardless of where you fall on the spectrum, your code should still be readable. That means being reasonable with the naming of functions and variables, indentation, and overall structure. Comments should not need to explain what the code is doing. However, comments can explain why the code is doing what it’s doing. For example, you may be perusing your old code to find a strange set of arithmetic. Although you see the math, you don’t know why it’s there. A helpful comment could be # this ensures the number is within the valid input range.
Again, I’d love to hear others’ thoughts! What workflows do you use to develop add-ons?
over 2 years ago
Spencer! Thank you for taking the time to make this. I found it very helpful to read and contemplate about how I'm writing my addon right now. I really appreciate it.
I had the worst time getting blender to recognize my multiple files in the same directory... Thanks for the link on that.