The Dark Art of Performance Optimization
Optimization is a tricky thing to do, difficult to quantify and can have troubling impacts on your whole project. This doesn’t mean you should never optimize your code, workflows or any other part of your operation. Quite the contrary, you should, but take this article as a signpost warning of dragons.
One should never attempt optimization too early in a project. This is because over the live of a project, everything changes. Frameworks get updated, architecture changes, bugs are found and fixed, dependencies evolve. During the early stages of any project there is a lot of change and honestly more important things to do. You need to get features out and in front of customers as quickly as possible. Performance is important, but the fastest working feature that a customer can’t use is worthless.
Because of that I leave all non critical optimization till as late in the process as I can. What is critical optimization? Basically if something is too slow that it fails (i.e. timeouts, crashes, etc) or doesn’t meet requirements (i.e. a stock trading app that takes hours targeted to day traders) are critical. Optimization along the lines of (it takes 2 seconds and I feel that’s too long) is not in the critical path.
Take this image on the right from Thenuzan’s article on the SDLC. Performance evaluation is the last step in the process. Personally I believe Performance Evaluation should occur after 3 and again after 4. Testing/QA can easily find some performance issues.
A lot of software projects are never really finished, till they are finished. Go-Live dates get pushed, betas go on forever, scope creep occurs, pivots happen, etc. But you can usually easily identify when the framework, architecture, tooling and environments are no longer in a state of flux that optimization activities can then proceed.
There are a lot of people out there who feel that performance should be feature 1 in everything they do. Which it’s always important to keep performance in mind context is important here. If your Amazon, your checkout/shopping cart performance is vital. But the vast majority of software developed is not the Amazon shopping cart and will never reach that level of importance to your company.
Perspective is important because performance optimization is complicated, time consuming, fragile and has wide ranging impacts to your project.
Optimization is a balancing act. Imagine you have 1 point you place in the triangle above, the more architecture and maintainability you have the worse your performance, but too far to performance the less maintainable and architecturally sound your project will be.
Due to the impact the process can have, it’s important to measure current performance, and set reasonable goals. “Making it as fast as possible” is not a reasonable goal. Personally I use a standard rule of thumb that no operation should ‘lock’ the UI for longer then 3 seconds. But you can’t assault your user with 3 second wait times either. Imagine a form with lots of drop downs, every time you select something from a drop down your user has to wait 3 seconds, you will have pissed off users. Measuring and goal setting will give a clear definition of success, which when going down the dark pit that is optimization is required.
Everything done during performance optimization is a trade off. Your trading off maintainability and/or architecture for that performance gain. Because of this changes related to the optimization process should be reviewed by at least a couple other developers. It’s easy to get tunnel vision when optimizing, and outside perspectives to the impact to others having to work in the same code base are vital. The tunnel vision can cloud other paths to achieving the performance gain, what you think is ‘the only way’ is rarely that.
Here are my 4 musts for performance optimization:
1. Set realistic performance objectives based on need/use case
2. Measure and Monitor Performance throughout the process
3. Share, present, get feedback and refine approach
4. Know when to start and more importantly when to stop
Performance is important, but always remember it comes at a cost. The more performat your app is, the more complicated the code usually becomes, the more difficult it is to maintain and the more convoluted the architecture is.
If you’re a First Responder or know one check out Resgrid which is a SaaS product utilizing Microsoft Azure, providing logistics, management and communication tools to first responder organizations like volunteer fire departments, career fire departments, EMS, search and rescue, CERT, public safety, disaster relief organizations.
