Study: Software Quality Problems Show Need to “Shift Left”

SOURCE: SQS Software Quality Systems

In the private sector, software errors are causing both financial and reputational damage.  A study by software quality specialist SQS identified private sector businesses in the banking, retail and mobile sectors, more likely to suffer software malfunctions than public sector organisations.

The SQS team analysed two years’ worth of news reports about software and computer failures, covering 964 stories and 245 UK-based organisations. While the level of public sector computer-glitch reporting remained constant during 2011 and 2012, reports on private sector computer problems have tripled.

The retail sector was the most error-prone in 2011, with 21 per cent of all stories, while mobile followed at 10 per cent and banking and local government at 6 per cent each. In 2012, reports of banking sector computer failures rocketed to 61 per cent of all stories, followed by retail at 7 per cent, mobile and education at 4 per cent, while local government claimed 2 per cent of glitch stories.

Stephen Fice, MD of SQS UK, commented: “Software is increasingly more complex at a time when private sector firms are under pressure to attract new customers through innovation, and offer a high level of service to retain existing customers. Private sector companies are adopting new technologies at an unprecedented pace, and price-sensitive consumers expect high standards or will search out an alternative supplier. In comparison, public sector IT budgets have been cut or remain static, so new IT systems are less likely to be adopted and consequently fewer glitches are reported.  Also, the public sector is more likely to adopt tried and tested technologies, which are less risky.”

New trends such as mobile banking, online retail and cloud computing are being adopted by the private sector to remain competitive.  However, new technologies resulting from these trends are not always compatible with existing IT infrastructure, and this is where major problems can occur.

“Inadequate requirements gathering continues to be a major reason for poor quality of delivered software projects, and is driving the need to “shift left” and introduce software quality earlier in the software development lifecycle.

“All IT projects should have quality and testing built in from day one to reduce the likelihood of errors. No longer does a testing expert have to scroll through lines of endless code, instead testing is being automated and virtualised.

“If software quality doesn’t move up the private sector’s agenda, then we can expect more reports of IT bugs as technologies become more complex,” concluded Fice.

Software quality and testing should be delivered by qualified, in-house testing teams or specialist outsourced consultants that have the breadth of knowledge to help reduce software problems.

Examples of software glitches ranged from banking consumers being unable to access accounts to duplicate payments; while retail sector glitches include online customers overcharged, online deliveries failing to make it for Christmas, festive hampers failing to arrive, Scottish customers paying up to 70 per cent less for their shopping, iPads for £49.99 and bicycles for £1.

Public sector errors include students sitting the wrong GCSE exam, car park exit barriers only operating for Spanish speakers, approximately 164,000 pensioners charged at least £800 each after being sent the wrong tax code and car park lights being left on for four months racking up an estimated cost of £28,000.

About SQS Software Quality Systems

SQS is the world’s leading specialist in software quality. This position stems more than 30 years of successful consultancy operation. SQS consultants provide solutions for all aspects of quality throughout the whole software product lifecycle driven by a standardised methodology and deep experience in various industries. Headquartered in Cologne, Germany, the company employs approximately 2,400 staff. Along with a strong presence in Germany and the UK, SQS has further subsidiaries in Egypt, Finland, France, India, Ireland, the Netherlands, Norway, Austria, Sweden, Switzerland, South Africa and the US. In addition, SQS maintains a minority stake in a company in Portugal. In 2012, SQS generated revenues of 210.1 million Euros.

SQS is the first German company to have a primary listing on the AIM (Alternative Investment Market) in London. In addition, SQS are also traded on the open market of the German Stock Exchange in Frankfurt am Main.

With over 7,000 completed projects under its belt, SQS has a strong client base, including half of the DAX 30, nearly a third of the STOXX 50 and 20 per cent of the FTSE 100 companies. These include, among others, Allianz, Beazley, BP, Centrica, Commerzbank, Daimler, Deutsche Post, Generali, JP Morgan, Meteor, Reuters and Volkswagen as well as companies from every other conceivable industry.

Top tips for IT managers to avoid software errors

1. Have all stakeholders, including testers review requirements.
2. Manage the business risk – understand what the system does for your business and what failure will cost you.
3. Use your test manager to explain how close you are to a safe release, if you’re not ready, why not, and what do you need to get there.
4. Look at your history and find the three top mistakes you will never repeat.

General tips to avoid software errors

1. Build quality into your organisation – get everyone involved.
2. Test your software early and continuously to detect errors during development and before roll out.
3. Have independent testers check the software developed by your team.
4. Rerun tests when your software changes, using test automation where you can.

Research methodology – overview

1. We searched through the archives of 35 UK major national and regional newspapers, from the Financial Times to the Yorkshire Post from 1 January 2011 – 31 December 2012.
2. We searched for terms, both plural and singular relating to software errors, for instance, “computer error” and “software failures”.
3. We removed all international stories. We de-duped and classified the information.

Sample errors by sector

Banking

1. Consumers unable to access accounts
2. Banks demanding mortgage customers make up repayment shortfalls caused by a computer error
3. Debit card system grinds to a halt. Consumers had their cards declined at tills, while business clients were unable to accept payment
4. Card payments taken from accounts twice, affecting more than 700,000 consumers

Retail

1. World famous London retailer failed to get Christmas hampers out in time
2. New iPad, which retails from £399 – £659, offered at one online retailer for £49.99
3. Christmas food shopping delivery slots cancelled by retailer
4. Pregnant mum orders £44 worth of toys, but £225 taken from her account, leaving her unable to buy children’s lunches or pay for transport to her antenatal appointment
5. Leading retailer seeks permanent staff in return for jobseekers allowance and expenses due to IT error in job ad
6. Retail website offers £85 bike for £1
7. Shoppers charged five times the amount they spent on grocery shop

Mobile

1. Bugs in the operating system of a best-selling phone from leading US manufacturer reduced battery life
2. New maps application failure – a map reader looking for Crewe will arrive at the new destination of “Wrexham, Crewe”
3. One of the biggest breakdowns in service suffered left millions of smartphone owners across Europe, the Middle East and Africa unable to send or receive emails, browse the internet or use messaging services
4. Telecoms operator  forced to apologise to almost 8 million customers after an “embarrassing” network failure led to a 24-hour blackout of its service across the UK and Ireland

Local authorities

1. Housing benefit payments to thousands of people  delayed due to a backlog in processing caused by a computer glitch
2. Highways engineers faced a “deluge” of potholes needing repair after a broken computer caused a severe backlog.
3. Approximately 3,000 social care customers charged for an eight-week rather than a four-week period following an error in its billing system
4. Street lights were left on at a park 24-hours-a-day for four months – at an estimated cost of £28,000. Visitors started complaining about the round-the-clock illuminations
5. North of the border council services ground to a halt by a massive computer crash. The fault left thousands of council workers unable to log onto the IT system for a whole day

Postal services

1. A computer glitch hit post offices bringing counter services to a halt during the busiest day of the year. The national computer system crashed, leaving Christmas shoppers unable to post presents, tax their car, pay bills or access their savings

Sports/Olympics

1. Ukrainian Mariia Pomazan’s Discus gold medal was downgraded to a silver due to a computer error
2. A Dereham grandfather travelled five and a half hours to carry the Olympic Flame after a computer confused his Lancashire address for one in Norfolk
3. 20,000 people mistakenly informed they had booked Olympic tickets when they hadn’t
4. The official Olympic countdown clock stopped only a day after being unveiled in Trafalgar Square
5. Computer glitches led to poor television coverage of the men’s cycling road race as BBC presenters struggled to work out who was leading because GPS data containing the competitors’ progress was delayed

Healthcare 

1. Staff at a Wiltshire hospital had to hone their linguistic skills because car park exit barriers only communicated in Spanish
2. A family on £350-a-week was turned down for a medical card because a computer error said they exceeded the weekly allowance limit by almost £8,500
3. Ten women in a large West Midlands borough missed out on repeat smear tests after their initial samples were unusable. They should have been recalled to have to have the test re-done but were not because of a computer glitch, an NHS audit revealed

Tax

1. Hard-pressed pensioners were told to pay at least an extra £800 each. Around 164,000 people who retired received a letter demanding the payment after they were given the wrong tax code. Those affected were told the mistake was due to a “computer error”.

Central government

1. High-tech eye scanners that were meant to slash queues at airport passport control were scrapped. The iris recognition checks were brought in at an estimated cost of £9 million, claiming they were capable of processing travellers in as little as 12 seconds. But after 385,000 passengers submitted their details, the scanners were ditched at Birmingham and Manchester airports, and were slated to vanish from Heathrow and Gatwick after the Olympics
2. Contractors were unable to mend a £280,000 electronic voting system after it crashed
3. A new computer system caused chaos to records on cattle tested for bovine tuberculosis in the South West – resulting in farmers having herds incorrectly placed under movement restrictions

Education

1. Some students caught up in a university’s admissions were mistakenly told they had successfully achieved a place. The university said it would contact hundreds of students affected by the computer glitch
2. An IT glitch led to more than 150 primary schools reporting problems with the new literacy and numeracy computer-based assessments
3. 50 pupils sat the wrong GCSE exam due to a computer glitch. The Academy pupils who were given the wrong GCSE biology exam paper were allowed to re-sit the exam