Creating too Big to Fail Apps

April 1, 2009

I can’t help it. I read Steve’s post too big to fail on brip blap and I immediately apply its principle to life as a developer. The focus is about breaking down “too big to fail” life events into smaller more manageable components that are less likely to wreak havoc.

Too Big to Fail

How is this relevant to anything in the world of software development?

1. System Design

When designing systems, separate components by logic functionality. Systems that are responsible for continuous database queries, should not be tied to a web site that serves content, such as in Simulate a Windows Service using ASP.NET. While this idea may be good in theory and for any shared hosting environment when you can’t install or run services on the server, the potential that a deadlock or frozen thread occurs will increase. If that occurs, you can kiss all of your active user sessions goodbye.

2. Application Design

Separate application logic out so that when one component gets changed, it doesn’t force a complete rewrite. Websites and web services should not tie directly to the data layer. Design N-layer systems (read the differences between N-Tier and N-Layer applications) so that there are no direct dependencies on lower-level technologies being used.

Maintainability in separate components increases as the number of layers does. Take this in moderation though, too much loose-coupling in your layers, leads to overkill and needless complexity.

3. Class Design

When designing classes, follow the Single Responsibility Principle (SRP) of the S.O.L.I.D Class Design Principles. If your class has more than one reason to change, then it may be “too big to fail”. The below example helps:

public class Worker

{

    public string GetTaxRate() { }

    public void Save() { }

    public string GetId() { }

}

The above breaks the SRP because any minor change required to one of the three purposes of this class causes the whole class to change:

  1. The process to calculate the tax rate of the employee
  2. Database Schema that matches to the Worker
  3. Employee Id that is used. SSN today, numeric number tomorrow

Any single change impacts the class which is responsible for three very different purposes. While this is not “too big to fail”, the class is “too big for its own good”.

Advertisements

Usability – Does it Exist or is it Just a ‘Theory’?

March 12, 2009

Have you ever heard a non-technical computer say that they are having computer problems whether it be viruses, pop-up porn ads, slowdowns, or network problems? Often they do not know where it came from. Hearing this have you immediately thought, “I never have any of those types of problems. This person must just not know how to use their computer.”?

As developers we don’t have these problems because we understand how a computer should work. We know what will get us into trouble and can easily identify warning signs. We also troubleshoot ourselves out of situations. In short, regardless of how usable the system, mentally we understand what needs to be done to get what we need.

Is this how it should work? Certainly not. We cannot take users for granted, hoping that they understand how to use our application. A user interface should handle system and application commands that a user wishes to run. If our users knew they had to call process A, click button B five times, and cache the response before the desired result would occur, there would be no need for usability (or us). Users would be smarter, and perfectly capable of troubleshooting their own bugs.

Such a perfect world does not exist and it is our responsibility to ensure that the software we release to our users doesn’t have any potential holes or pesky bugs in it. We must focus on making our applications more user friendly so that it is obvious when they perform an action that endangers their computer, or that those actions never occur in the first place. As developers how do we do that?


Usability Testing in Progress

Usability

So what can be done? Do we preach that usability testing is necessary before production? No. That music falls on deaf ears. Besides, who wants to give up valuable project time or money to test an application that is already behind? Even when usability testing is scheduled and performed, there is such a short amount of time left in the development schedule to fix any recommended changes.

My recommendation is that usability testing should come as an iterative and cyclical process throughout the development life cycle. It should not be considered usability, but instead usability development. Team members should be aware of it from the point requirements are defined to the point that the final piece of code is written.

This should be done by pushing the below responsibilities onto all team members:

  • Do not let anyone test their own code for the final round of testing
  • Concentrate on understanding the needs of users early
  • Perform early and continuous testing. This can be done through peer developer testing or lead developer testing core functionality

Soft Coding

February 26, 2009

Do you know what is potentially worse than hard coding values into your source? Soft coding. Why? First a definition:

Soft coding is the act of writing code so that it depends on external settings such that behavior may change without updating the source.

The reason we hard code is because we are lazy and don’t know any better expect that given conditions will never change. Soft coding is born out of opposite reasons. Soft coding is intended to accommodates business rule units subject to change so that application modifications can occur without forcing code pushes or deployments. Soft coding is often seen in the form of configuration files.

Used cautiously soft coding can provide ways to make code become dynamic (switching logging formats, web service URLs, using reflection to create different business objects, etc.). When used in excess, soft coding complicates code by over-abstraction and develops into an anti-pattern.

<BizRuleEngine>

 <RedirectToImageProcessor>

  <if expression=inparameters[0] > inconditions[0]>

    <inparameters>size</inparameters>

    <inconditions>value</inconditions>

    <processor ref=BizRuleEngine.ImgProcessor, BizRuleEngine />

  </if>

 </RedirectToImageProcessor>

</BizRuleEngine>

Below are a few of the pitfalls that I’ve run into with soft coding:

  1. Increase of complexity: Developing dynamic code based on configuration values is essentially equal to writing another new language. The problem here is that we try to account for every possible scenario by making the code capable of doing things that will never be done. This results in spaghetti code and it just gets messy.
  2. Development complications: Code is written in an IDE because it can interpret the grammar and check for syntactical errors. Debugging is a breeze in most cases. Again, we are writing a new language and the problem is that we often don’t put as much time and effort into this portion of it. A lot of room is left open for loopholes and unaccounted conditions, while increasing the possibility that an unexpected exception will occur.
  3. End-user error: MY opinion is that this is a big issue for CMS sites. Developers spend lots of time building out the site, making it template based, and hand it over. The new owners then proceed to make content changes that bring the site to its knees.

The better your software is when produced, the less customization it will really need and the less users will find that they wish to change it.


Over Complicating Your Job

January 15, 2009

Elaborating a bit more on my last post on dealing with poorly designed applications, as developers we often find ourselves in sticky situations where it consumes more time to fix something that should take little to no effort at all. I will be the first to admit that I have caused my own fair share of these problems.

Transformer

Below are some of the most trivial solutions that I’ve collected from my own experience and samples that I’ve read. These can be fixed by better development and coding standards:

1. Over Designing

Ever meet someone too smart for their own good? There is a fine line that smart people often cross when writing code. Once this line is crossed, the code they produce becomes an inversion of their intelligence. You can identify this when they constantly refactor code that you’ve written to be more efficient, dynamic, generic, and re-usable. I’ve seen developers try to make .NET generics generic.

public class BaseGenericWrapper<T> : IGeneric

{

 

}

Please do not invent something that will synergize the evolution of social networking when all you need to do is build a blog. Yes, some things are prone to change (content, configuration variables) so don’t hard code those. At the same time, if your team is the only one using the application and are expected to maintain it, then make the solution straight-forward and simple. They will thank you after they make an emergency fix.

2. Code Complexity

Nothing can be worse than a lack of unit tests when making modifications or upgrades to someone else’s code. With poorly written code, it is difficult to determine if the change being made will break anything or not. Even worse, is the change that has to take place in the spaghetti function that has 800 lines of code in it. One line gets changed and it seems as though you just witnessed the program in self-destruct mode.

Maybe an analogy will help here:
If I were asked to find my keys in my own house, I would have an idea where to look. The kitchen, the bowl by the front door, etc.. This is because it is my house and I am most familiar with it. Now imagine I asked you to find some keys in my house. Where would you start? Would you start looking for a utility bowl that said “keys” on it, would you look in drawers throughout the kitchen, pockets in a coat?

Sometimes our code closets are left in a state of disarray for someone else to take care of. When they need to change something, do they know where to look? Is your code clean and self-descriptive so that they can easily understand what each function does to prevent a change in the wrong place?

3. Over Commenting

As a student going through school, we were taught to comment our code. I was told to provide pre-conditions and post-conditions as well as a summary and a description of every parameter to all public/private/internal methods. You’ve seen it happen and I’ve seen it happen, another developer (or yourself) comes along and makes changes to the way the code works, adds a variable or a parameter or two, and then doesn’t update the comments. At that point our entire easy code universe is out of sync.

What about saving yourself the time and instead of writing code that requires comments to explain how it works, you write code that is completely legible and flows together so that it requires no additional comments? Take a look at this snippet that I grabbed from Jeff Atwood’s Coding Without Comments and you’ll be able to see what I mean.

This:

r = n / 2;

while (Math.Abs(r – (n / r)) > t)

{

    r = 0.5 * (r + (n / r));

}

Console.WriteLine(“r = “ + r);

becomes this with a comment:

//square root of n with Newton-Raphson approximation

r = n / 2;

while (Math.Abs(r – (n / r)) > t)

{

    r = 0.5 * (r + (n / r));

}

Console.WriteLine(“r = “ + r);

and can be simplified to this:

private double SquareRootApproximation(int n)

{

    int r = n / 2;

    while (Math.Abs(r – (n / r)) > t)

    {

        r = 0.5 * (r + (n/r));

    }

    return r;

}

See how much easier it is to understand? I don’t have to understand that that the complex code calculates a square root since the method name explicitly states it.

Having said these points, there are many things that you can do to simplify code and make it more developer friendly. Be careful not to waste too much time breaking code down and finding re-usable solutions. I’ve walked my self into many a corner trying to come up with creative ways from having to write a ton of GUI code and handle dynamic layouts. Just keep it simple and the code will speak for itself.

Do you have any tips for keeping code simple and straight to the point?
How do you feel about the way that our developers that come straight out of school code? Are they taught properly?


Dealing with a Poorly Designed Application

January 1, 2009

Auto Mechanic: Sir, I’ve replaced the spark plug on your car.
Customer (sarcastically): What’s the damage?
Auto Mechanic: $600.
Customer: $600?!!!! How? It was just a simple fix!
Auto Mechanic: In theory, it should have been. But the way the engine was built and the placement of the plug made it very difficult to fix. We had to tear half of the front end off in order to get at it.

Unintelligent Design

I hope this never occurs to any of you, but it definitely puts the problem of inheriting a poorly written/design application into perspective. “If only things had been done right the first time, changing it would be so much easier”. How many times have you heard that mantra? Hindsight is 20/20. We should focus on how we deal with these messy situations instead of complaining about them. Here are a couple thoughts:

1. Relax, look around, and determine what is going on.
Before you go haywire on your project manager and threaten to quit the company, put a positive spin on it. Remember you were asked to be the expert and fix someone else’s mistake. Make sure you really determine what the problem is. Is the problem the fact that this was written poorly, or is the problem because you have to maintain it?

2. Recognize what you can change.
If the application that you were asked to fix is completely abysmal, make sure that you don’t dig yourself into a deeper hole. You obviously don’t want to work on this application longer than necessary (unless you are a glutton for punishment). At the same time, question why things were done the way they were. Reasons may have been entirely political and you may have absolutely no control over design. Don’t focus on those scenarios, focus on situations where you can help the next developer who needs to fix this.

3. Make sure you know what you are talking about
.
Ensure that this is a poorly written application because it uses bad design methodologies, not because someone else wrote it. In explaining the problem to your peers and other developers, avoid a negative attitude. They are just going to see you as someone who complains, not someone who solves problems. When solving the problem, don’t create change for the sake of it or because you think you can do it better. The time for design has passed, this is development. Unless you were asked to do a rewrite, do a fix.

4. Lose your fear.

If you’ve got the last point down and you know what you are talking about, lose your fear. Dive right in. Express what is incorrect with the application to managers and see how they respond with it.

5. Get away from it.

I solve my most difficult problems away from the computer. Sometimes it happens at home, sometimes during dinner, and even in the shower. You know the deal. If it isn’t an urgent production issue, give yourself a 24 hour mental break. Think about something else. This answer will come to you tomorrow and you will laugh that the solution is not as complex as you expected. 

6. Attack it in units
.
Don’t bulldoze the whole project. Take notice of the systems’ components and break the problems down into smaller units. Ask yourself how you can make these pieces perform more efficient. If you can’t get any more juice out of it, then make the code more readable and understandable. As you further break the code down into individual methods and components you can create unit tests to ensure that each portion is acting as intended. This will benefit future developers so that they can easily make changes and ensure that they aren’t breaking existing functionality.

7. Use this as an opportunity to educate others!
Always come away from every experience with something you have learned. It may be something you learned not to do, but either way you know better. I will repeat, hindsight is 20/20. Use it as the chance to help others know better too. Find a junior level developer and challenge them. Ask them what they think the problem is and see if they are capable of producing a better solution.

I read somewhere, developers have a high tolerance for working with ugly code (as long as they were the one who wrote it). No one wants to openly admit that they created a code catastrophe. However, really what we should do when we write code is not put others in harms’ way like we have been. In order to do that, we should make sure that the code we write can be broken down into as many chunks as possible. Complex problems require simple solutions. There is always the developer with an ego problem who tends to believe the opposite. They think that complex problems require tricky coding. Sometimes those developers are so concerned for writing a generic solution to the problem that they go above and beyond what the requirements ask for and in the end make it more difficult to troubleshoot or fix the problem.


Top 8 Reasons to be a Software Consultant

June 16, 2008

After experiencing my first year in the workforce, one of the things that I really want to address is helping other (soon to be) college graduates, particularly software engineers, find out what they want to do with their degree once they are in the real world. From my experience, my employment director’s job consisted of emailing list after list of companies who were looking to hire a typical entry-level programmer. The problem is, that I never knew what those positions offered or what the expectations were. Details about the job and the employee’s responsibilities were very vague. On the other side of the fence, from the recruiting calls that I was involved in last year, it seemed that most students didn’t know kinds of developer opportunities were available to them once they graduate. One of the big topics that I tossed around, was the concept of becoming an in-house developer versus a consultant. That said, I’m going to list the Top 8 Reasons to be a Software Consultant below and hopefully that will provide students with some ideas for what to do with their college degree.

  1. Expertise – Let’s face it, the fact that the client didn’t use any of their own developers to run this project, or still needed your help, just goes to show how much your opinion is respected. Most consultants focus on particular products or industries, so when you are brought in to help, you’re expected to know what it takes for the project to succeed. As you focus more on one particular industry, you can apply experience learned from a previous client to the next one.
  2. Variety – Most of the projects that consultants work on (I said most!) tend to last a few weeks to a few months. Some of them take up to a year. Depending on how many projects you are involved in with the client, or what industry you are in, you could be at a client for more than a year. Chances are however, that you continually work on different projects. As an in-house developer, you will most likely continue to maintain existing applications or develop the same set of technology.
  3. Networking – You will, without a doubt, always work with different people. The exposure that you get will help you gain valuable contacts and develop many relationships. Working with so many different professionals will allow you to get exposure to many different opinions which will help you form your own.
  4. Broadened Horizons – Whether or not you consistently work in the same industry, it is inevitable that some of the clients will want to do something leading-edge and use never before seen technology sets. The advantage here, is that you will develop many different skill sets in a short amount of time, at the client’s expense. Where else can you work and be told that tomorrow you will begin working with mobile text-messaging or writing with the new Microsoft MVC Framework before it is even released?
  5. Improved Communication and Presentation Skills – When you work long-term within your own project team, you really work yourself into a comfort zone. Consulting takes you out of that zone and places you in front of people you don’t know. The next client you present to could be the big break that you need. You will continually fine-tune your communication and presentational skills so that you are able to effectively sell a service or prepare a proposal for a potential client. As a software consultant you’re not just a code monkey while others make decisions. You are brought in front of a client and asked your opinion. Each opportunity will improve your skills and as time progresses you will have no problem presenting to large audiences.
  6. Sense of Accomplishment – For me each project that I work on and complete is another “notch on my belt” and another accomplishment. As a consultant you will get exposed to so many opportunities that you’ll run out of space on that belt and begin forgetting about all of the different projects that you worked on. Another aspect is that being exposed to many different technologies helps you evaluate previous experiences and decide what you could have done better for the next project.
  7. Challenge – One of the big challenges about consulting is that you will spend most of your time working or integrating with systems that you didn’t previously build. In some cases the amount of hoops that you have to jump through to get existing systems to integrate with your new product can be nauseating, however once you get all of those systems in place together you can really relish in the sense of accomplishment as you roll your latest project into production.
  8. Relief (between clients) – This doesn’t apply to all consultants, but one of the best things about working for a client is the roll-off period when you get to go back to the home office (or your home for that matter) and work on honing up new skills while you wait for the next project to roll in. Until that client comes in, your focus could switch and you might begin directing your efforts internally or research a new technology that you’ve wanted to learn.

Backwards Compatible Web Services

May 31, 2008

I do a lot of work in distributed environments using service-oriented architectures. One of our biggest projects just launched, and at the time only had one client (Client A) consuming the web service. Now as the number of clients grows, the test is on to make sure we built the service as intended and any number of clients that need its functionality can use it without requiring major changes.

We’ve already run into a problem trying to make our second version of the web service backwards compatible, and unfortunately it is going to require the first client to update to the new WSDL. In the first version of the web service, one of our objects had a string property called AccountNumber. Now the Client B is integrating they require this existing object to support an array of AccountNumber. We had a couple of different options for how to approach this so that Client A was not affected:

  1. Add the AccountNumbers property with no other modifications.
  2. Add the AccountNumbers property and mark the original AccountNumber property with the Obsolete attribute.
  3. Make the AccountNumber setter point to the first index in the new AccountNumbers property.

Option 1 is the easiest, but for a new client offers the most confusion as they may not understand the difference between the two properties. The responsibility is also on the service now to determine which property was intended to be used or if both are supplied, which one takes precedence. Option 2 which I thought would be the best option, as that is the route to take when designing class libraries, actually surprised me. When marking a property obsolete, it is actually removed from the WSDL thus any client who provides that value is providing additional information that the WSDL doesn’t define. At this point, that was not going to work as well. Option 3 seemed it would also work good until I ran some unit tests and was having problems nullifying out the values. Once again the responsibility was up to the service to determine which value would be accepted if both happened to be supplied. What happens if the object was sent to the web service with the AccountNumber property set to a value and the 0th index in the AccountNumbers property set to null? The result depends on the order of serialization/de-serialization.

What option did we ultimately take out? We caved and dropped the AccountNumber property and implemented the new AccountNumbers property, requiring our client to update their version of the WSDL so that they could comply with our service.

What defines a web service as a backwards compatible web service? This happens when:

  • new methods are added to the service
  • restrictions are removed (access restrictions, mandatory values are now optional, etc.)
  • optional attributes and properties are added
  • attributes are changed but their type is compatible with the previous type (number to string)

Herein lies the complication with building web services and making sure that you design so that further down the road you don’t require all of your existing clients to change to adapt to your updates. Fortunately for us, there was only one existing client, so this change is much more possible.