Product Story: A Missing Link in Product Development
Things that are forgotten…
Suppose you, like me, are interested in history and technology. In that case, you must have read or seen documentaries about what happened in terms of technology during the First and Second World Wars.
I want to compare part of the Product Development Process with one of the engineering issues of World War II.
During World War II, aircraft engineers worked to improve their endurance on the battlefield. In short, these people were studying the planes that had returned from the battlefield. And all the points that were more damaged were considered prone to further strengthening.
But the point was that, in fact, the vulnerabilities of the aircraft, which caused several planes to be lost on the battlefield and never returned, were never seen. Because the planes that were shot down never came back so that they could be inspected.
Now consider the software product provided to customers. And we, as the Product Owner of this product, are managing the development of this product. With the valuable efforts of the development team, QA and designers, we review and present new features and fix existing bugs. We include the technical debts in our sprints and eliminate them as well.
And we think to ourselves that everything is going well. But this is not the case. We are still losing our planes on the battlefield. And by mistake, we’re fixing the weaknesses of the aircraft (Our product) that have returned to base.
Abraham Wald was the one who realized this error in examining the problems.
And to strengthen the planes, he did this:
- Check the returned planes
- Identify defects that bullets have hit
- Strengthen the parts that have not been shot.
The armor, Wald said, doesn’t go where the bullet holes are. It goes where the bullet holes aren’t: on the engines.
In the product development process, where do the bullets hit our product?
Just when used by our customers and users.
Are we present on the battlefield (where and when our product is being used) to test its performance?
No. We are waiting for the customer to face a bug and to be notified by the CS team. (At our base, we are waiting for the planes to return to check on them)
Do we only review and improve areas where problems have been reported to us, or do we address the issue in-depth?
Unfortunately, because some of our software bugs and many confusing points of our product are fixed by the CS team for customers, we do not realize them, and as a result, we can not improve our product.
Especially in the case of the last question, Part of the CS unit capacity is currently spent on:
- CS explains the obscure and confusing parts to customers and users
- CS solves the number of users’ problems that arise due to the complex steps in the product or its confusing design.
- CS is doing the work of customers and users. Because they have run into so many problems, CS has helped them every time. Instead of doing those things themselves. When they encounter an issue, they go to CS and ask them to get the job done.
- The CS only notifies the product and development team of complex problems that it has not been able to resolve. And from this point of view, the two product and software development teams are in the dark.
When do we realize this problem?
A large part of the CS team’s capacity is allocated to these items, but when the product team does not know about them? Just when a huge workload occurs for CS for whatever reason.
We realize that the CS team has been doing routine work for a long time that should have been referred to the product and development teams. And these problems exist in the user experience or the software’s functionality, which should have been investigated and resolved sooner.
QA team role:
This was implemented in the Air Force before software product development teams. An expert/team or even a department that is responsible for QA.
The role of the QA and the test pilot is rooted in the Air Force. And it was the test pilots who were later transferred from the US Air Force to NASA, not the operational pilots.
But there is a problem here.
As we know, QA colleagues, with the same tests and checklists prepared for the regular operation of the software and the function of all its parts, test it before releasing it.
After a while, testers also rely on their subconscious to execute these checklists. And they will no longer test it as a complete and comprehensive product that they encounter for the first time.
There is another big problem.
A tester from the QA team will never use our product like a real customer. It also has nothing to do with those customers' intellectual, cultural, and educational backgrounds.
Usability Testing vs. User Testing:
Sometimes to resolve biases that arise in the QA team and its members. Software teams conduct Usability Testing events. In my personal opinion, this is a very, very good thing. But as you know, Usability Testing is a significant difference from User Testing.
In Usability Testing, we ask the user to review and use the solution we have provided to their problem as a software product feature. So that we can finally see his behavior while using the product to find its drawbacks and even positive points.
But in short, during User Testing, you will see that the user examines your solution and the essence of your product. Not only is your solution a feature of your product, but your whole product may not be the correct answer to the user’s problem.
Based on the experience and articles in this field, we can say that the first step in achieving a more favorable situation is changing our perspective on software product development. We do not just have to look for problems based on the symptoms and offer solutions.
Instead, we must be careful that the symptoms do not distract us from the more important goal. For example, in the case of airplanes, The goal was to be able to reinforce the weaknesses of the aircraft by identifying them, but in practice, we were improving the mediocre points. And the breakpoints of the planes remained the same, and they were still vulnerable.
I suggest you read the How Not to Be Wrong: The Power of Mathematical Thinking book. It’s helpful for these kinds of biases.
As the product owner, let’s not forget that we are 100% responsible for the product and its success. So we have to follow the customer home and witness the use of the product in practice. Remembering the valuable efforts of the development team, QA and designers, none of us are customers. But we try to understand them.
Sometimes we do this with data. Sometimes we hear their voice through the CS team, and sometimes on the battlefield of our customers’ business, we need to see them, and how they are using the weapons we make for them.
I suggest you watch the movie starring Tom Cruise.
In the film, an army general asks a representative of a military equipment manufacturer to wear the same equipment and be present on the battlefield.
Feel free to comment below. Please hit that clap button to help others find it if you enjoyed this article.