The S Curve

Outsourcing The S-Curve

In my experience, outsourcing software projects (whether onshore or offshore) has more often than not failed to meet delivery or cost expectations.  I have seen this phenomenon when working with major consulting firms as well as with smaller boutique firms, whether the project teams were located in the same building or a dozen time zones apart, and I have seen it for projects of all sizes and complexities.  I assert that a key contributor to this failure of expectations is due to the fact that almost all of these projects have had non-repeatable characteristics (new features, new paradigms, new teams, etc.) and we fail to account for (or sometimes do not want to accept) the factors related to the S-Curve when we pass on the responsibility of these project to a third-party. 

Of course, this failure to meet expectations is not to be equated with a failure to deliver the end result (a product or successful project), since competent teams and leaders find a way to make that happen.  But the costs and/or timelines often meet a significant mismatch in expectations. 

So why is The S-Curve so significant, especially when dealing with outsourcing?

I believe that through experience and daily interactions, we develop an implicit grasp of the functioning of the internal teams and adjust our expectations appropriately with regards to the processes, factors affecting delivery and eventual outcome of projects.  But none of this is possible with a third-party since there isn't the same amount of visibility (if any) into these factors.   This forces even a greater emphasis on identifying, quantifying and including the factors into the delivery plan.  This process is obviously easier if you are dealing with an experienced partner with an established team than with one that is about to bring a team together specifically for the sake of the project.

With respect to the type of factors to be taken into consideration for these projects,  refer to the article by David Anderson titled "The S-Curve Explained" (http://bdn.borland.com/article/0,1410,32411,00.html) in which he points out a number of factors that contribute to The S-Curve phenomenon both at the start (Team Formation, Knowledge Sharing, Tools, Environment, etc) and end (project integration, bugs, etc) of the project. 

While his focus is towards management of Agile Techniques on software projects, the concept can be generally applied.  You may even add or take away from this list based upon your own experiences and team maturity.  The point here is simply that these factors need to be applied to the plan and measured (however quantitatively feasible) throughout the project's timeline (and during post-mortems) to minimize the mismatch in delivery expectations.  Post-mortems would facilitate in identifying key improvements for the next phases of the project, if the relationship and work with the same partner are to continue.

Sanjiva, San Francisco, Oct 30, 2005

 

Software as a Knowledge Medium

Often in the context of managing software development and even the business around it, we tend to invest much more time and effort focusing on the process and mechanics of software delivery and much less on the content itself.  As a consequence, there is insufficient investment into the product definition (or elaboration) process and resulting unpredictability with regards to the success of the product in the marketplace (or its alignment with customers' needs).  While the organization focuses on the dates of delivery as the key milestones to achieve and celebrate, it fails to set specific expectations or measurement criteria with respect to what the product needs to achieve in the market or with customers.  I have rarely seen processes or operational structures in place that can 'directly' facilitate such measurements or accountability. 

Similarly, people are more apt to accept the failure of certain features in the marketplace ("customers and markets are changing too fast and product couldn't be delivered yesterday when we really needed it") than delays in shipment.  The exec responsible for feature/content of the product will often have a clear idea of what they could easily sell in the market today and even support the need to invest more heavily in engineering to achieve the "immediate" delivery needs.  But  they are less clear or committal towards even the near-term future.  In the absence of the sales team meeting its quarterly revenue goals, the typical response would be "if we had x features today, we could easily make the quota".  Of course, when those features are delivered, the target changes to the next delivery date on the roadmap (or so it seems, at least to me :))

As all this attention and pressure diverts to the engineering team, it is easy for them to feel that the organization is not taking the responsibility for developing enough of a vision or strategy (or a commitment towards this strategy) against which to focus its execution tactics.  The behavior of the sales and marketing teams is often viewed as 'deflection  tactics' to keep the pressure off of them.

Recently, I came across a presentation by Phillip Armour titled "Of Zeppelins and Jet Planes" in which he contends that software is a knowledge store and not a product, therefore, software development needs to be managed as a knowledge acquisition process and not a product production process.

While Phillip appears to focus on development processes and methodologies, I can easily relate the argument to changes in operational and organizational tactics and behavior to place a deeper emphasis on the decisions regarding 'what to build' rather than the obsessive focus on 'how' to build it'.   

This is not to suggest that there is always sufficient clarity with respect to the specific requirements that constitute a product definition or direction.  As Phillip points out, this is becoming more and more of a moving target, therefore, our methods must allow us to deal with such a moving target.  My point here is simply that this is something the organization must recognize and react to and not just the engineering team.

http://www.corvusintl.com/biograph.htm#Phillip%20Glen%20Armour

Sanjiva, Mallorca, Oct 21, 2005

The Art and/or Science of Communication

It is often difficult to appreciate the difficulty that we have in communicating with each other.  I presume that this is primarily because we take communications for granted and defer the responsibility for comprehension to the other party.  Conversely, we tend not to spend much effort in helping others understand the level of information that we seek from them to understand situations.   I feel this way because I have found lots of situations where people have struggled with ineffective communications and very rare instances where anyone has stepped up to focus on making it more effective.

People face communications challenges often and everywhere; they face them in relationships as well as in professional settings.  Communication becomes even more difficult when executive teams are assembled rapidly (in the process of re/building companies) and they need to manage their respective goals while attempting to influence the overall direction of the organization.

Within the context of a team, the difficulty in communications is compounded by a number of factors.  Some of these include:

• Personality and communications style (see note below on a Forte Institute profiling experiment)
• Cultural background
• Philosophy
• Experience
• Language and expression skills
• The chemistry (or lack of it) on the leadership team
• The trust level within the team
• The sense of security and overall support that each member on the team feels he or she has from the others
• The clarity of goals and objectives

These factors do not all weigh equally in influencing the degree of the gaps that exist within a team.  Moreover, I have come to realize that some of them (leadership, strength of the team and clarity of goals) can more than make up for others. 

Communications needs also differ from one individual to another.  While one CEO insisted on his team structuring their communications in the S-C-R (Situation, Complication, Resolution) mode ("we have x customers with y workload and are unable to support new efforts without z impact, therefore possible scenarios include …."), another one wanted the R-C format (ex: "just tell me that you need additional headcount to support the needs of new customers"; no additional details needed unless asked for).   

[In one company, the leadership team invested in the Forte Institute's Interpersonal Communications System (https://www.theforteinstitute.com/introduction/monograp.pdf)

and a consultant to help us appreciate the difficulty we were having in understanding and consequently working with each other.  Through this profiling scheme, it was determined that of the thirteen members on the team, yours truly stood alone in the top left quadrant (Dominant/Conformist -- implying someone who deliberates, is cautious and focuses more on the execution details and less on the big picture) while the other twelve were mostly Dominant/Extroverts -- more apt for the entrepreneurial types, not focused on consequences of their actions or execution details but more focused on the big picture and creative thinking).

Not to indulge into the details of Forte's science or mechanics, one of the things that became clear to everyone was that my deliberations over key technology decisions had often been mistaken for indecisions.  Similarly, being on the top left quadrant meant that I tended to focus a lot more on the execution details of specific decisions and recommendations and tended to relate our goals in the context of those details--clearly creating a mismatch in communications style with the others on the team.   

Beyond just appreciating our differences in personas and communications styles, I am convinced that there was sufficient depth and detail in such an investment to allow us to improve the overall effectiveness of our communication, should we choose to apply the needed effort.  We didn't and the results of this exercise proved futile and a wasted experiment.]

When faced with these circumstances, I have found that people will either resort to the comfort of familiar faces, choosing to work with peers and subordinates from prior organizations -- because the existing working relationship seemingly reduces the overall need for communications or start placing significant emphasis on details, facts and conclusions (and increase the overall quantity of outputs) rather than more qualitative assessment of situations. 

And this becomes even a bigger challenge when managing remote teams (regardless of their mandate, i.e. whether they are engaging in sales or development or any other activity).  Here the geographies, time differences, language and cultural factors became much more pronounced.  While we spent countless hours communicating status report, our understanding of the situation, status, risks and potential actions become increasingly limited.   Lack of effective communications here does not in itself lend to project failures but it reduces our ability to understand and manage the situation, thereby increasing the risk of potential failures.

In this context, I applied one simple technique with my team that worked quite effectively for the type of information that I sought from them, although it required trust and patience on both sides and mostly my needing to understand what quantity and quality of output I needed from them.  Rather than focusing on the output itself for various status reports, proposals, etc. that they would share with me, I spent more time on each line item asking them three questions:

1. Why are they telling me
2. Why should I care
3. What do they need from me (or someone else on the executive team)

As an exercise, we went through several presentations that they had for me and worked diligently through the line items.  The process was tedious and iterative but began to produce effective outputs.  It also forced me to focus on all the details and continue to provide them with relevant feedback so that they knew that I was focused on trying to understand the inputs.  And lastly, at the end of each session (or call), I would play back for them my understanding of the status or proposal.  To the extent that I could absorb it (in abstract terms), they got both encouraged and excited about their efforts.

What is the S-Curve

A few years ago, one of our board members had forwarded an article to me titled "The S-Curve of Software Development".  I have since lost the reference but I remember the overall message--that process introduction into a relatively young and immature development environment has some early costs with respect to the throughput of new functionality delivered over time, but the investment pays off as the scale and speed of new development increases as these processes are applied. 

This article was brief and its message fairly clear.  At an abstract level, it is also something that people have been trying to communicate for decades through various channels.  In this context, given a small startup with a young engineering team, it served to reinforce certain development paradigms and associated consequences on delivery plans and timelines.

For early stage software companies, the problem is multi dimensional and seemingly much more complex mostly because there is less clarity with respect to its overall mission and a greater need to be reactive, nimble and adaptive.  The problem involves more than negotiating or balancing the effects of process improvements on feature introduction.  When we are still attempting to define our market, products and customers, the clarity and definition of features, their priorities and complexities are all in constant flux.  It is analogous to the 'mathematical' challenge of trying to solve one equation with many unknowns.    

So while everyone finds themselves at some point on the S-Curve, there isn't a simple equation or recipe that plots the optimal path through it.  In my view, it has mostly involved applying intuition and experience and factoring in the maturity and leadership capability of the executive team and their ability to commit to a specific direction for the organization. 

I had dutifully passed this article on to some of my peers, needing to drum up support for our challenges in the development cycles.   As always, such communication generates occasional sympathy towards our plight but fails to set any concrete expectations or affects any specific behavioral changes, decision trees or execution paths in the executive ranks.  In retrospect, I realized that communicating abstract messages was not a productive exercise.  Not only were my messages, such as this one, lost on the team, but none of us always understood each other's challenges well enough (perhaps because each one of us was often communicating in similar ways).  Such reflections have since made me more aware of the need to deliver more concrete messages that are specific to the actions, decisions or behaviors that I am expecting to influence within the team and the organization. 

Through this blog, I hope to share some of the challenges, successful techniques and more importantly--failure scenarios that I experienced (and managed) while navigating through the S-curve.

Where are you on The S-Curve?

-Sanjiva,  Oct 15, Pt. Alcudia, Mallorca