Fox in the Henhouse: The Power of Independent Software Assessments

Software ReviewHave you ever been in a situation where you were tasked to critique or review a fellow associate’s creative work? 

How would you feel about going to an art gallery and providing criticism directly to the artist? 

It’s not easy, right? How secure would you feel about your software if I said that this happens in your organization every day? For many organizations, the quality of their code is directly tied to peer reviews and constructive criticism.

Sociologically speaking, it’s tough to be in any of the above situations, where you have to provide constructive criticism to a peer. I truly admire those who can remain objective and factual during any peer review, be it about art or software. I know from my own creative experiences, it’s unpleasant to take your work in for review by a peer or superior. It’s difficult not to be defensive and take constructive criticism personally.  So, in many cases, the colleague reviewing your work may gloss over some of the errors to prevent hurt feelings and awkward workplace situations – which, in this case, means your software is suffering.



There are a number of assumptions organizations make about their software, which they believe is proof that their software is high quality and secure. Let’s evaluate these assumptions, shall we?

“Tools simplify the problem” – We all wish this were true, but the reality is that while most software tools do find quality and security defects in software, it is at the consequence of finding voluminous false-positive data and, more importantly, errors of omission that we don’t find out about until it’s too late.

“Code peer reviews are always performed” – Even including our clients that follow a CMMI Level 3+ process, we don’t have one of them that would answer the following questions with an absolute yes: 

  • Are peer reviews performed on all of the code produced? 
  • Do you know when they were performed, the result and trending information from the mitigation plan? 
  • Do all of your engineers use software tools to assist and do you know how the resulting data is managed to a measurable result? 
  • Are the results socialized with peers, superiors and upper management to proactively understand quality and security risks?  

“We never shortcut our process” - No one company or development organization can defy the laws of software physics to bend its code around space and time. However, this doesn’t eliminate the pressure of trying to deliver new functionality and products to market before the competition to capture more market share. Therefore, shortcuts are taken, with the belief that we will circle back and do it the right way when we have more time, which we never have.    

“Software engineers are super humans!” – It’s mission impossible for software engineers to do everything, but that is their responsibility (right?), including:

  • Memorizing 1,000s of software standards by language.
  • Reviewing not only their own code, but also their peers’ code, in time to make a release date.
  • Sifting through 1,000s of false-positive data instances produced from software tools to identify the three percent that are real killers. 

“Engineering peers can overcome the pressure of reviewing code” – I’m not convinced this is fair. I have no doubt our engineers try their very best to be constructive, but they more than likely also hold some criticism in order to not damage a relationship. Also, many of our clients do not invest in training their engineers in the proper method of reviewing code.



First, there is no silver bullet for ensuring you have quality code, but there are solutions that provide key capabilities to any software organization. As both an advocate and provider of independent reviews of software, we have seen dramatic impacts on organizations. The most common feedback is appreciation for the truth in the data produced.

When you hire an independent, they do not have a vested interest in the outcome of the service, other than the integrity of the process and ensuring that the resulting data is preserved at extremely high levels of assurance. The collection of massive amounts of software risk data, leveraging numerous software technologies, is an eye-opener to clients on the power of technology hooked into a well-defined data management process. Correlating these identified risks to quality, security and performance concerns against multiple standards, across multiple language spaces, yields the organization economies of scale in effectiveness and efficiency that they are unable to achieve using internal resources under pressure from market, customer and competition demands. 

The most dramatic impact we have seen in organizations is the effect that socializing the resulting information has in improving the collaboration between executive and technical teams. Both audiences trust the information because neither produced it. This does not completely eliminate the hurt feelings in pointing out risks and flaws in someone’s code.  However, at a minimum, it opens up the lines of communication because the conversation is based on facts and not opinions. It’s amazing to see the dynamics of the room change from hostile to “We’re in this together” once they have a common framework of understanding the data. From personal experience, we oftentimes hear a sigh of relief from engineers once they understand the amount of work required to muscle the data down to actionable results, happy that they didn’t have to do it.      

It’s 2014 and software has been in our society for 30+ years, powering just about everything these days, from our homes to our cars. We should be past the common myths of believing our own engineers truly have the time to review code the right way – and the courage to remain objective. Perhaps with recent examples in the news of several companies’ stock prices taking a hit from software glitches (i.e. Twitter server crashes) and cyber attacks (i.e. Target), it might be time to consider another approach, offering superior data integrity to make critical business decisions in the best interest of our brands, customers and stockholders.       


Rob Cross
Vice President, PSC

Written by Rob Cross at 05:00



Subscribe to Our Newsletter
Join over 30,000 other subscribers. Subscribe to our newsletter today!