On April 26, 1994, China Airlines Airbus A300 crashed, taking 264 lives. This tragedy can be traced to a software bug. Recounting other software related mishaps, Guru99.com also reports:
- In 1985,Canada’s Therac-25 radiation therapy machine malfunctioned due to software bug and delivered lethal radiation doses to patients ,leaving 3 people dead and critically injuring 3 others
- In April of 1999 ,a software bug caused the failure of a $1.2 billion military satellite launch, the costliest accident in history
- In may of 1996, a software bug caused the bank accounts of 823 customers of a major U.S. bank to be credited with 920 million US dollars
Every one of these horrendous events might have been avoided with more diligent software testing. These are not isolated events, and still happen to this day. Mistakes cannot be eliminated from the human equation. And software is still a very human equation.
A human has to design the software in the first place. They design it for other humans to use in ways that the designer may not have originally intended, on hardware that the designer may not have been able to completely predict, in circumstances far beyond the designer’s control. Let’s take a look at some of the challenges you may not have considered:
Fragmentation
Fragmentation is a word that is casually tossed about in tech circles, usually as a pejorative against the Android operating system. It refers to the unmanageable diversity of product types and attributes in the Android ecosystem.
This image represents the current state of what is known as Android fragmentation. OpenSignal is the company responsible for compiling this information. Their data shows 18,769 distinct Android devices downloading its app.
These devices all have different hardware specs, different screen sizes and resolutions, and running different versions of the Android operating system. Currently, there are 6 versions compared to 2 on iOS. While the latest version of iOS is on 9 out of 10 Apple devices, only 2 out of 10 Android devices sport the latest version of the operating system.
Just by looking at these differences, it is easy to see why popular applications come out on iOS first. Developers are able to test their apps against every iOS device configuration as there are only a handful. On Android, this is simply not possible. Developers have to pick and choose which devices they will target. Even if the iOS and Android projects were started at the same time, Android testing takes a lot longer, and will be a lot more difficult and expensive.
Applications intended for corporate and government use require an even more rigorous standard of testing. In this sector, fragmentation is not an excuse, but an opportunity. The challenge is not just software testing, but software testing at scale. For consumer-based ventures, Tricentis functional testing is one of the solutions serving the virtualized E2E niche. This market sector needs to:
- Test case design and risk coverage in one tool.
- Save time by simulating real test environments.
- Test complex systems without scripting.
Enterprise software testing tools go well beyond the needs of the average software house that just has to make sure their latest updates look okay on the two most popular screen sizes. Regardless of the market segment, a highly diverse hardware landscape has to be accounted for.
Software Conflicts
Fragmentation is low-hanging fruit. But it is far from the only challenge for software testing. Let’s say you have designed and coded a perfectly functioning application. You test it in every conceivable way on your developer device. Because you want that device to run smoothly and predictably, you have nothing on that device beyond the bare essentials to run it.
Unfortunately, that is not how the general public will be experiencing your application. They will be running it on machines that have all their personal data, and all of their downloaded apps. Consumers are also not the most security conscious people in the world. They will download anything free. They leave Facebook running in the background non-stop. They never close apps. Oh, and they might be running a rooted OS to enable free tethering.
The point being, real app use by real people is a messy and unpredictable affair. Developers cannot just test their apps in a cleanroom, on clean devices. Sometimes it takes months of the product being out in the wild before it can be truly tested.
Feature Compatibility
Smartphone operating systems are extremely complicated. Even iOS, known for its simplicity, has a great deal of non-intuitive complexity. This feature creep is inevitable because each year, all companies are compelled to add new things to keep the OS feeling fresh. This can be fun for consumers, but a nightmare for developers.
Take the accessibility features in iOS as an example. Now, responsible developers have to make sure that the built-in screen reader can read the app. Apple provides the resources for this. But not even all of Apple’s applications are 100% compatible with their screen reader.
The Notification Center is activated by a single-finger swipe down gesture, while Control Center is a single-finger swipe up. What if your app already uses those gestures for something else? Too bad. You have to test for that and make the appropriate changes. TextKit allows for dynamic resizing of certain text elements. How does that effect the usability of your app which was crafted for a fixed text size? Good luck with that. And that is just a few considerations for iOS. The challenges is exponentially worse when testing for additional platforms.
Comprehensive software testing is the single most important step a developer can take before releasing their software into the wild.
