Managing complexity in Software Development
Page is part of Articles in which you can submit an article
written by owen on 2008-Sep-26.
In programming, sooner or later you will come up with a solution to a problem that is somewhat "complex". The said solution may be advantageous to the solve in various ways. The thing to note is that you should ensure that this complexity is "contained". Far too often software developers let a complex solution to a simple problem seep in to the body of the program itself. As a result making the entire program unnecessarily complex. Miracle solutions are proposed to increase productivity through various catch words that end with "-ibility".
For example the index page of this website is probably the most complex page in the entire website. Yet the website itself is quiet simple. The index page was not as complex as it was now. Originally it was a series of IF conditions that directed the visitor to a requested page based on various parameters. IF PAGE == ABOUT THEN LOAD ABOUTPAGE. It was really REALLY simple. One day I realized that if I could find a uniformed way of determining the next page then I could replace all the IF statements with a controller or MVC (as they call it now a days). The new solution required careful thought, because adding complexity to a working system should never be approached lightly.
As with every line of code or function created there are things to consider;
- How is changing this page going to make the other existing pages BETTER.
- How is changing this page going to make the other existing pages WORSE.
- how is changing this page going to affect the quality of code as a whole.
- Will changing this page make the other pages more complex or less complex.
These are important points to consider when dealing with complexity. You should always beware of things that may unnecessarily complicate the entire program, only offering short-term benefit but long-term headache. If I had suddenly woke up and decided to change the index page to use a random MVC without considering how the actual MVC would affect the complexity of the program itself then I would have made a grave mistake which leads to people discontinuing software a year after it was implemented.
All things must be considered in light of the short-term and the long-term when developing software. A piece of software that starts out complex will not get simpler as you develop it.Keep the simple parts simple It will only get more complicated until it collapses under the weight of its own complexity. In the same light something very simple can be complicated by a misplaced bottle neck in the system that was implemented on a "whim". Implementing features on a "whim" or ad hoc without considering its benefits is the fastest way to complicate what was once a simple program. Keep your eye on the important parts of the program to ensure that it remains simple and that any complex algorithms and concepts are "Functionalized" away from the program's core purpose.
In the case of my index page (which loads all the other pages) it may seem to be against what I just said - a very complicated concept deep within the core of the application. The thing to note is that the index page itself is a FUNCTION. What it does is completely independent of the program/website. It can easily be replaced because the other pages are not affected by it, nor thou or both shall go with him. hmmm. It is a function. An important function but a function none the less. The other pages on the site are unaware of its existence and so are not affected by any low level changes that I make to it. The sub pages behave as if they are in a vacuum, unaffected by the index page. It is not a typical MVC that seeks to solve a problem by controlling/limiting structure rather than providing a genuine solution to the structure problem.
What I could have done and what most people do is make the index into a class. And then enslave other pages into a "house of cards" situation. Instead of isolating the solution to a problem of IF statements - the programmer ends up complicating the entire program in order to simplify it. I am not saying that wide spread refractoring is not necessary. I am saying that the implementing complex solutions to a problem should not be allowed to complicate the program as a whole. Find a way to solve the problem without creating thousands of little problems/new workflows in the progress. Alot of this is explained to more dept in software engineering anti-patterns
Now for the code.
Here I will list elements that add complexity when writing programs;
- IF THEN ELSE Statements: Avoid having a catch all "else" statement. Usually it leads to logic bugs and poor clean up.
- Avoid unnecessary class implementations: There is nothing more annoying than having to do a "new" every single time you want to use a base class. Its pointless. If you only need one instance at a time why is it a class? Make a single function that manages itself. Always ensure that you know the "why" behind everything you program so that you can better make it when it needs to change.
- Avoid giving yourself future work or more per-module work: Now I understand that some people like the whole "house of cards" thing but really you should avoid it. It is easy to copy development documents off the internet and pass then off as your own but hard to implement something that you have no experience in using.
- Avoid doing the same thing over and over with slight per-module changes: This is a another sign of a train wreak waiting to happen. Sooner or later you are going to have to re-factor all those slight changes and you will realize that 3 lines at the top of every module was simply there to make you feel productive but added nothing to the program.
- Avoid tight coupling of you classes. This often occurs by using checked exceptions to force calling processes to handle them. Hence causing every calling program to be tied to a useless/decorative exception tree of choas.Avoid eating you own dog food: I cannot stress this enough.
- Avoid fragmenting processes: There are times when you may be tempted to create many small functions for the purpose of modularizing/simplifying a process. This usually happens when you have so much code that its difficult to really keep track of what is going on. In such a case avoid creating singleton functions that look like functions but are infact big GOTO statements. These singleton functions usually have to be called in a specific order and a specific custom object has to passed to them. Its better to just keep it all in one function with good documentation. Ten random singleton functions are neither helpful nor do make the program easier to read/manage.
- Avoid classes with too many setup functions: Some may think its a good idea but classes should manage as much of its internal processes as possible. Avoid;
Its better to create a function called ans = calculate_test(10, 'test'); Not only do you have less lines of code (per call) but the test calculation process is encapsulated.
c = new calc class;
ans = c.calculate();
- Avoid try-catch-throw logic: if a function is boolean it should return true or false. It shouldn't throw random exceptions to the calling function. Try-catchs are often used as a lazy form of IF-ELSE. Situations like this should be avoided for the same reason as mention before.
- Avoid bandaid errors, hiding errors and non-specific error messages. A system is not secure if its errors need to be a secret. Having clear error messages allows for quickly identifying problems during the development and testing phase and evening during the life of the system. Searching through log files for errors that could have been easily displayed to the user is a waste of time. Critical errors can be missed that are hidden for years in log files and catch statements. The last thing you want is someone to be calling you in the middle of the night about some program that you wrote 5 years ago. Assume All errors are fatal!
- Assume all errors are fatal. Often times errors are ignored or logged at lower levels of the program and the program is allowed to continue. This is often done to prevent the program from crashing. PROGRAMS CRASH its what they do when something is wrong. It is more important in program processes to identify WHEN a error occurs as opposed to the order. Log files are good for error order but poor for edge case event errors.
If your going down the wrong road it never too late to turn back. A poorly written program is only good as long as no one knows about it. You can throw as much RAM or HARDWARE at it as you want. Sooner or later it will get replaced by a off the shelf software that is probably JUST AS BAD OR WORST THAN your custom developed software. And the software you wrote will be just another skeleton in your closet. The main difference will be that the off the shelf software has learned to manage its complexity, increase flexibility while meeting user requirements.
Often times there is nothing you can do but to just run with what you have, make as much documentation as you can and pray for the future developers.
permanent link. Find similar posts in Articles.
You shouldn't have to tell people to avoid eating dog food
if they don't heed your warning, well, nothing could be done for them
by Gods Child 2008-Sep-26
programming is like creating dog food. We do it all the time. Programmers at times have to eat the dog food created by other programmers. It happens either out of ignorance, inability to find better dog food to eat or some kinda force. All this is ok and common place in programming.
BUT when a programmer starts eating his/hers own dog food it creates a "circular reference", writing code for code sake, running on a treadmill. Or Building a ladder while you climb it, following your own tail, writing lots of useless code, giving yourself more work because it appears to be benefitial when it infact it is simple scafolding. Its hard to explain without getting all technical.
by owen 2008-Sep-27
Not all errors are friendly and surely not all errors should be bubbled to anonymous users.
In your coded examples above encapsulation can be achieved in both instances. However I do appreciate the fact that often times what is needed is a well written function that will perform what is needed.
On a whole your comments are valid...know what you want to do before you do it, else you might paint yourself into a corner. Then having to find a way to climb the wall to get to the door where u started.
by rhode 2008-Sep-26
Try Catch block are not bad. In fact there are quiet usefull. But when combined with exceptions can be lead to a world of pain. There is only one good reason to use a try catch block and that is to catch errors thrown from interfaces that you donot control or that you have no means of "pre-checking" for possible errors.
The horror comes into play when you start using them in code as a form of logic or "process control". Programmers often end up calling functions willy-nilly without following the proper procedure and using a try catch block to print a stacktrace.
You start creating custom exceptions that you bobble up to calling procedures at random. And these procedures start using different exception types to control process flow. Since you can only catch one exception at a time you start having a case of exception bloat where you start creating more and more types of exceptions for different situations and doing different things when a certain exception is thrown.
Check this article Best Practices: Exception Management. It never really says that exceptions are bad but if you look at the best practices section IT OUTLINES A LONG LIST OF THINGS NOT TO DO as opposed to things you should do. Which is a tell-tale sign of a pitfall.
For a real world coding example of the misuse of exceptions/trycatch check out Handling Alternate Business Scenarios. The author of that post has the right concept but eventually starts eating his own dog food instead of doing the proper checks BEFORE calling the function. He has a function called DO_UPDATE which VOID. But instead of making the function BOOLEAN manages to double the complexity of his program while ignoring basic programming practices and starts throwing exceptions.
by owen 2008-Sep-27
Interesting post. You said "The last thing you want is someone to be calling you in the middle of the night about some program that you wrote 5 years ago." I said, "Amen, brother!". The thing to say to them when they call is, "So, is your maintenance contract up to date? If not, you're phu-ked! If it is up to date however, I'll call you in the morning.".
by Mad Bull 2008-Sep-26
I consider putting dead kittens in people's mailboxs and running away to be unprofessional and irresponsible. So I make sure that has at least 6 lives left.
by owen 2008-Sep-27
By the way, Rhode wrote his comment while I was writing mine. I didn't see it, so didn't say anything about it in a previous comment. I would like to see each of you argue this topic (try, catch, etc.). So far I am inclined to agree with Owen. Rhode, what makes the try, catch approach better to any other?
by Mad Bull 2008-Sep-26
try catch serves one good purpose "if (network is down) print [error network is down] ", instead of crashing the entire program and forcing a reboot. But like most everything in programming, it can be misused in many many horrible ways.
by owen 2008-Sep-27
Interesting post indeed. Felt like i actually learned something.
by Stunner 2008-Sep-27
wow!! i am with Stu, *did this from a psp *
by bobby 2008-Sep-27
Owen, what is your definition of, "Complexity?"
Surely that's the starting point which will allow you to gather all your recommendations into a coherent strategy (fine recommendations though most of them are).
Some of your recommendations are heading towards encapsulation theory here:
by Ed 2008-Oct-08
I find it strange when people don't have dates on the things they write. But complexity however you define it often increases with lines of code. You start with a single line and every line you add after that is added complexity.
I wouldn't recommend a strategy because often times the only way to solve a problem (in a short space of time) is by writing a 300 lines of code with bad variable names and "on the fly" arrays.
by owen 2008-Oct-08