The S Curve

Learning from Rachel Dawes

"But it's not who you are underneath, it's what you do that defines you"  -- Rachel Dawes in Batman Begins      

         

This catchy quote from Batman Begins had me a bit puzzled when I first saw the movie.  What does she mean by that, I thought?  But it stuck.  If you have seen the movie, you will understand why.      

                  

And then in a totally different context, while trying to submit zAgile to various search engines and business directories, it became somewhat clearer.  The problem I encountered had to do with figuring out where to categorize zAgile.  Yahoo's taxonomy seemed pretty dated and perhaps severely constraining.  It shouldn't be so difficult, I thought.  But it was.  Why do I have to fit zAgile into some pre-defined category?  Why can't I just say 'what it does' and have 'what it is' inferred?  The problem with Categories is that an individual may belong to more than one category and the categorizations are not necessarily permanent.  Categories definitely cannot be used to define what something 'is'. 

          

So it seems that we are constantly constraining ourselves by putting 'things' into specific categories.  When you have to put a book on a shelf in a library then obviously you need to make some hard choices.  But in life, what something 'is' ought to be defined (and inferred) based on its attributes and relations to other things.   

                            

In a somewhat obtuse, indirect and 'Heideggerian' sort-of way, maybe Rachel was trying to say the same thing.  Her quote curiously made me realize the key difference between taxonomy and ontology.  We are so used to dealing with taxonomies when developing applications that the power and flexibility of ontologies are difficult to grasp and leverage.  While ontologies often take the form of taxonomic class hierarchies, they are not constrained by them.  Therefore, rather than struggle with categorizing something, we use an ontology to define it using its characteristics (or attributes) and relationships to other things.  And all that it 'IS' is simply inferred.      

      

And that is all what Rachel was trying to tell Bruce Wayne (so I believe, anyway).         

         

-Sanjiva

'If Onlies' of Application Lifecycle Management

In a recent blog, Carey Schwaber (Forrester's Senior Analyst focused on Application Lifecycle Management) points out some of the unconquerable hurdles that we face in software engineering -- the perennial 'If Onlies':

• If only requirements were complete
• If only there were enough time to test at the end of the life cycle
• If only everyone followed the process
• If only development efforts were predictable
• If only metrics were comparable across projects
• If only our tools worked together
• If only we weren’t constrained by legacy
• If only we could reuse our prior work
• If only software were easier to evolve

It is easy to resonate with this list.  Almost every organization faces these challenges.  It is also easy to add so much more of 'if onlies' to this list from our experiences. 

There is also another dimension of 'if onlies' with which we struggle the most -- a dimension that involves visibility, i.e. 'If only we know'.  For emphasis, I will map each one of these to Carey's list above.  In my experience, lack of visibility has been the key source of problems that we encounter (and she exposes) in Application Lifecycle Management. 

 

If only we know...

• whether requirements are complete, when they are complete, when they changed and why they changed.  Product Managers, Business Analysts, End Users -- all play a key role in the definitions but most of the time we are building software to meet the changing needs of a business or customer(s).  While the delivery cycles need to be rapid to allow the business to stay competitive, what's explicitly needed isn't either always clear or it is changing rapidly.  Not to mention the fact that we don't always know how much we need to know until we get into the delivery cycle. 

• what will be tested, how will it be tested and how will it allow us to measure product quality.  When requirements aren't well known or are fairly dynamic throughout the delivery cycle, QA often resorts to ad-hoc testing.  There is no way to know when 'that' cycle could be completed.  Good QA teams are able to plan very precise execution of test plans and test cases that map to known product features/requirements.  And if they are efficient and creative, they can even automate some of this -- but this level of execution is practically seen mostly in maintenance cycles -- and rarely in new product or feature development.  Therefore, not only do we not know how much testing is needed, we often do not know what specifically is being tested.  This becomes more critical in large organizations that need to connect these dots to address their stringent auditing, compliance and traceability needs.

• how to effectively capture, disseminate and track the methodologies and processes the teams are using.  This is a difficult one but when you have distributed teams (offshore, outsourced, remote etc.) -- most managers struggle with establishing or controlling specific process that are being followed by their distributed teams.  For starters, there isn't an easy way of capuring or tracking an organization's methodologies or processes.  Any suggestions of a process or methodology typically exists in Powerpoint slides, Word docs or wikis but does that mean it is being executed with precision.  Since process tracking is manual, it relies on a lot of walking-the-hallways or 6am phone calls to India (or Ukraine or …) to track.  Add to this the complexity of disparate tools used by different teams and it becomes very difficult to follow the process. 
• how to leverage an organization's best practices towards future projects, how to tie them across teams and how to evolve processes and methodologies through effective post-mortems -- all in an effort to improve overall predictability of delivery. 
• what and how to report with respect to project progress and how to get that information. We often do not have control of what we are trying to track mostly because the information available to us is so limited.  What do we track -- the process or project status or quality or completion or compliance or …?  This gets crazier when you have teams in different geographies with different processes and qualitative metrics used in their reporting. 

• how to integrate information across teams and tools.  Every team manager excels at hand-picking tools and implementing processes that are effective for their teams.  But he or she is not able to incorporate those across other teams.  And breakdowns in ALM most often occur during hand-offs.  Managers take solace in knowing that their teams performed effectively when 'inputs' were predictable and controlled, yet because of their 'siloed' mentality, they fail to coordinate and deliver towards the overall success of the project. 

• how to effectively capture knowledge needed to deal with 'legacy' software.  Legacy here doesn't necessarily refer to stuff developed in the 60's.  If an organization radically shifts its technology and architecture between releases then even software developed in the previous release can quickly acquire the 'legacy' status.  If likely it was developed using RAD and Agile techniques, perhaps not enough was captured to describe the functional or technical feature set. 
• how to create a knowledge source that integrates 'what' was delivered with the 'how' and 'why' -- to allow teams to leverage that knowledge and 'reuse' previous practices towards future work.  This is critical when you have offshore teams involved that have annual attrition rates of 20-30%.  If a third of your team disappears each year, they aren't likely to know what was done before and therefore how best to 'reuse' it.

• any relevant strategic direction of the product or technology (coming back to the overall goal or vision of the organization and project) at the level of those involved in the trenches -- to control any evolution of the product or service.  Well crafted and planned iterations allow for controlled evolution but the overall lack of visibility into the big picture still takes away from the overall effectiveness.

                                     

Of course, this isn't just about adding to the problem set (to address 'The So What').  There are 'alternate' 'transports' (using Carey's metaphor) to address even these 'if onlies'.  zAgile's Solutions provide the 'game changing tactics' by addressing each one of them.  In this blog, I wanted to provide some of the things that we have attempted to tackle with our approach to Intelligent Application Lifecycle Management. In subsequent blogs, I will provide details on how a software engineering organization may apply zAgile's Solutions to address these chronic pains.

                                                

-Sanjiva