Resource Center

Blogs

11 Tips for the beginning Software Architect

Lately I have been thinking about the role of Software Architect. The title has always bothered me. I have seen it used too often by people who started out as programmers, kinda sucked at it, and through political skill and the well timed repetition of three and sometimes four letter acronyms, managed to removed themselves from the nasty world of coding into the far safer domain of the Powerpoint box and line diagram.

Still, despite the bad reputation brought about by poseurs, good Software Architects are essential to the success of complex projects. Though we, as an industry, have become quite adept at programming in the small, designing large systems remains a mix of art and science with results heavily dependent on the skill of the designer. To produce good designs, Software Architects have to combine solid understanding of computer science fundamentals with good taste and creativity. Additionally, Software Architects often are responsible for selling their technical vision to opinionated, insubordinate and generally troublesome Engineers, the more talented of which are also the least likely to listen. It is not an easy job. So, as a small contribution towards the advancement of the art and science of Software Architecting, here are my 11 top tips for beginning Architects.

  • Learn to communicate. When I was in grad school, we often had experienced Engineers come and try to talk some sense into the students. The Engineers were a pretty diverse bunch, in ages, backgrounds and experiences, but the one thing they all seem to agree on was that communication skills were a big deal, even in Engineering. At the time I did not quite see it that way. No, I thought, what matters, is raw technical talent. Anything else is just a bonus. Over the years, after I worked with people who were smarter than me, but which were unable to express their thoughts clearly, I have changed my mind. To be able to accomplish almost anything of significance in Software you must work with others. And you cannot do that unless you learn how to communicate effectively.

  • Do not take Architecture documents too seriously. All Models Are Wrong. Some Models are useful. In most mature Engineering disciplines we create and use models. We use these models to reason about the properties of things we want to build, before they actually exist. We do this in Software as well, with UML, ER diagrams, data flow diagrams and so on, but unfortunately our models suck. They lack precision, formality and completeness and they offer little or no useful feedback to the designer. These flaws should make us wary of investing too much effort into producing design documentation. Perhaps one day we will have better models, or maybe our programming languages will become powerful enough we do not need them, but this is not the case today. Today, for better or worse, the real Architecture of a system emerges from and during the coding process. So, do not waste too much time with long, detailed Architecture documents. Do a rough sketch, good enough to allow basic planning, and get coding.

  • Keep your programming skills sharp. An Architect must first be an outstanding programmer. Software Architecture consists of moving between high and low levels of abstraction, between cloud diagrams and network protocol specifications. An Architect that lets his technical skills deteriorate cannot reason clearly about his designs and has trouble understanding the essential feedback that comes from the implementation work.

  • Do not let fear rule you. Fear is the mind killer . All Engineers in a project, but most of all the project Architect, must have a clear mental model for how their systems work. This mental model should be declared courageously in the code, using assertions to state the invariants assumed, trapping only exceptions you expect to happen and in general, stating clearly throughout the code the assumptions under which the system runs.

  • Avoid religious choices. Do not think of yourself as a Java Architect or a .NET Architect or a LAMP Architect. If you do, you align your fate with that of organizations or movements over which you have no control over. Study the fundamentals, and be promiscuous in your use of technology. You are paid to solve the problems of your client, not to promote personal bias.

  • Seek simplicity. In anything at all, perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away. Human brains were not designed to program computers. To have any hope of building correct systems, an Architect must ruthlessly eliminate unnecessary complexity.

  • Seek beauty. Beauty in Software Architecture is just another name for a combination of simplicity, power and fitness to a purpose. When you work on a design, try to make it beautiful. Trust your aesthetic sense and it will guide you to better designs.

  • Use numbers. Whenever you can, wherever you can, use numbers to guide your design. Once you have attached numbers to a problem you gain a tremendous insight that you can leverage to make the right tradeoffs.

  • Learn to discuss technical issues objectively. In such an immature field it often hard to tell how good a design is. As a consequence young, bright Architects many times develop an exaggerated sense of their own skill and become very territorial when discussing alternative approaches. Recognize this tendency and actively fight it.

  • Stand your ground. If you are the Architect for a project your are the ultimate responsible for its successful implementation. Do not let rogue Engineers sabotage the project Architecture. Do not let Project Managers or clients impose constrains you know are impossible to meet.

  • Read. A lot. All the time. There should always be a technical book on your desk. And you should actually be reading it.

Pedro Pinto

The Perils of Big Design Up Front

I used to work for a trading firm that operated in the derivatives market at Chicago Board Options Exchange. Derivatives trading firms make their money exploiting the theoretical equivalence relationships that exist between related financial products. Some financial products, like stocks and options on those stocks have equivalence relationships, that is a certain number of stocks has, at a certain point in time, the same theoretical value as a set of options on those stocks plus some cash.

Starting around the late 80's, sophisticated traders began to apply these formulas to actual trading. Trusting on the ability of their formulas to determine the real value of a financial product, the traders would buy when the market price was below the value predicted by their models and sell when it was higher. More often than not, the real world prices would revert to the theoretical values predicted by the formulas, and the traders would capitalize on their (informed) gamble. Many people became very wealthy doing this sort of arbitrage and today, twenty years later, no one trades options without using some variation of these formulas.

Somewhere around 2000 I was charged with building a trading client, an application what would run on a handheld computer used by the trader on the exchange floor. The application would be fairly simple; all it had to do was display the current theoretical values for the products the trader was interested in. But there were some interesting constraints, most importantly the unreliable and slow wireless network available to the handheld and the relatively underpowered CPU. I was relatively new to the company and wanted to show of my Architect skills, so I put a lot of effort in the architecture. It took maybe some 4 weeks to do the design and 6 months to implement it. But the effort showed: the system was a thing of beauty. A multi CPU server would calculate the theoretical prices incrementally, i.e. by paying attention to the type of input change and re-doing only the minimal amount of computations necessary. The server used multiple threads, each controlled by a state machine. Each state in the machine had defined pre conditions (things that had to be true for the state to be entered), invariants (things that were always true in the state) and post-conditions (things that had to be true upon exit of the state). Because of the careful design the system and despite the usage of multiple threads and complex, high-performance requirements the server software was highly reliable. I was very proud of the result, until we were a couple of months into production that is.

As we progressed, some details started to bother me. The central server software was reliable, but invariably something would happen to the hardware or the network or the calculation code and the server would stop calculations. And every single trader in the company would stop getting updated values, costing us thousands of dollars a minute. The carefully designed incremental calculation state machine turned out to be unnecessary as the vast majority of the input changes triggered a full recalculation. The communication code was not robust enough and although it worked well against the steady state of the network, it would not react properly to the spikes in bandwidth consumption and connection drops that would happen when the market heated up.

In other words, the Architecture was beautiful, and the implementation was high quality but ultimately the system was inadequate. The Architecture addressed the wrong problems and missed the real ones. This had a direct impact on the company's revenue and so over the following years the Architecture was incrementally evolved to match our growing understanding of what the real issues were. After maybe 3 years our systems had finally caught up with our top competitors in performance and reliability while still holding an edge in maintainability and re-usability.

This initial failure made a strong impression on me. I had the right Computer Science credentials, was an experienced designer and had worked diligently and intensely to design the system Architecture. Yet I had failed. I did not think lack of talent or training was the culprit, as the same team was later able to deliver a good system. The root problem was ego. I had lacked the humility to admit I did not understand the problem well enough to implement a complex solution. A typical case of Big Design Up Front. I had made a complex, big design, before I did any coding and deployment of any part of the system, that is before I had a real understanding of the problem.

This very expensive lesson has stuck with me. These days I am very reluctant to commit the resources to do comprehensive Architecture upfront, unless I really understand the problem I am trying to solve. Real understanding is not simply a matter of being more thorough when you collect requirements. There is often no way to identify and investigate every possible scenario at this stage. Instead, real understanding of the scenarios that really matter, is best obtained by implementing a subset of the system, one that solves some useful part of the overall problem, and deploying and observing that subset in action, out in the real world where it can bring value to the user. A working solution, even if it only addresses part of the problem, is worth a thousand PowerPoint slides, when it comes to communication, trust and understanding between users and developers.

For complex problems, in the absence of an extraordinary level of talent and experience in the problem domain, an incremental approach is mandatory. We must release value to the user in short cycles. At each cycle we must be disciplined enough to solve the problems we see, not the ones we speculate might happen in the future. And we must be willing to revise our assumptions and re-do our design in each cycle, as our understanding of the system improves.

Pedro Pinto

Why QUALITY matters: An example

Over the past five years, Quarksoft achieved a defect density rate of 0.3 defects per one thousand lines of code and on average, project schedule deviation was less than 15 percent. Initially, you may think these numbers don't impact you very much but as the following example will show, they are directly related to your success,

Let's say a team is going to develop a product that has 100,000 lines of code. Further, let's assume that the development team consists of five (5) engineers and that each engineer has a productivity rate of five (5) lines of code per hour (5 LOC/hr). When you do the math, at this level of productivity, the project will take 20,000 hours or 24 months (2 years) to complete.

Now, if the development team achieves a defect density of 4 defects per thousand lines of code (4 D/KLOC), the team will inject 400 defects during the life of the project. On average, a programmer will require 5-10 hours to correct a single defect so the team will require 2,000 to 4,000 hours or 3-6 months to correct the 400 defects. Six months is a 25% increase in the development time.

Contrast those results with a team that achieves a defect density of only 0.3 defects per one thousand lines of code. At this level of quality, only 30 defects will be injected during development effort and will require less than two weeks to correct. Only a three percent deviation from the total schedule.

So excellent quality, i.e., a lower defect density, will significantly reduce your product acquisition costs and get your product to the market faster, which will increase your revenue stream and profits. Most importantly, this paradigm shift will move you from just keeping up with your competitors to using your IT systems as a competitive advantage again.

By partnering with Quarksoft, you can spend the majority of your budget on new, leading edge developments instead of just maintaining yesterday's technology.

Stuart Locklear