A customer solution that does not produce clear value will not be used in the long run. Sooner or later the question raised will be if that is worth the money and as consequence, the solution will be turned off, leaving the vendor with a questionable reputation. A situation to be avoided from the start.
There are no simple problems. Everything that is simple, is no problem and solved already. That a simple solution is preferred over a complex one is undoubted as well. The problem is how to find the simple solution to a complex problem.
One approach is to build a product for a single use case. Very narrow, little to no flexibility. Which is fine if many users have the exact same problem but no option for Data Integration. Too disperse are the requirements.
The other approach is to implement the product with a subset of features only. The 80-20 approach. Which might be okay depending on how the cut is made. If a Data Integration solution can connect to 80% of the sources its value can be huge still. But if it supports only – let’s make a exaggerated case – only 80% of the source datatypes, then it effectively means it cannot be used for any source object at all, as any has at least one column with an odd datatype.
Worse, these things are found out only later in the project. The third approach is to hide the complexity with a very sophisticated solution. This creates the most value for the user but is also the most expensive solution to develop. Which hints towards the third value, the win-win.
The best is a healthy mixture of these three options of “simple”. Hide all the complexities behind automation and simple to use UIs. Plan the solution for the big projects but focus on the normal use case in the implementation. Add extension points so that things outside of the normal can be implemented. Too bad not all products allow that.
The idea behind Scrum is something we did for ages in the Data Warehouse space. Implement a Data Warehouse as quickly as possible, only then there is a common language and understanding between the various parties, the developers, the business data experts, the future end users.
The Scrum methodology expands that and puts the process into a solid framework, from a theoretical and practical point of view.
Just as a summary, the question behind Scrum is “Do we know exactly, down to the very detail, how the end product will look like? If not, why use a classic waterfall project management approach?”.
It is much better to have multiple iterations of the product, first only the core functionality, later add features and options and functionalities. In a way where development and business cost/benefits are well balanced, risk is reduced and the end solution is one product rather than a collection of features.
One of the harder parts is the communication between the developers and the business users still. This is where the Design Thinking methodology provides rails how to come to a common understanding.