Three insights

I would like to share three insights that have lately become a kind of guiding principles for my work on product/project management on analytics:

• Make it work, make it right, make it fast
• Separation of compute and storage
• Agile is not the same as fast

Make it work, make it right, make it fast

I first knew about this principle in a guide to data science job interviewing. Later on I read about its many interpretations in the mother of all wikis. The meaning for me is as follows.

Firstly write the skeleton of the algorithm, program or thing you are trying to build. Just the basics, with no real functionality, just to ensure that it compiles, runs or executes. Then write the logic in a simple, naive way, but that yields the correct answer. Finally try to improve on speed, space or any other critical performance parameter.

Apart from being in line with the avoidance of premature optimization, this way of operating helps in reducing uncertainty, building confidence, and showing early results.

Separation of compute and storage

Recently I read an ebook titled Creating a Data-Driven Enterprise with DataOps, and this principle appeared in a chapter about Cloud Architecture. Although initially it seemed counterintuitive, the more thought I poured into the concept the more sense it made.

In the cloud, storage and processing are decoupled, and the former is cheap and permanent while the latter is ephemeral and expensive. Compare this to on-premise analytics where both are coupled, permanent and limited.

The impact that this principle has had on me has been profound, even considering how recently I have discovered it, and it has helped me design, implement, and communicate analytical architectures in a convincing manner.

Agile does not mean faster

I have recently discovered the deeply rooted misconception that being agile means giving up on preparation and alignment of stakeholders, and just hoping that every problem will sort out itself during the process of development. This is utterly wrong.

The Agile Manifesto does not mention fast development a single time. Moreover, there are other references that explicitly state that agile does not equate to fast.

In my experience, an agile framework introduces routines and discussion environments where new requirements and feedback from users can be introduced in the scope of the project or product, faults can be detected earlier, and a valid outcome is ready at the end of a sprint. This, however, comes with an associated cost in time and coordination. Agile methods will make reacting to changes easier, but they will not replace preparation before starting a project, and they are not, by any means, an excuse to cherry-picking features to boost in the middle of a sprint for the sake of showing them off in a committee.

In the long run, being agile will mean being faster. This will be caused by hitting the mark more frequently, not by making individual projects faster. As a matter of fact, less projects are expected to be run with a higher level of resources. At the individual project level this is clearly slower and more expensive, but at the project portfolio or the product levels it completely makes sense, if the required time for the methodology to provide results is allowed.

Conclusion

These three insights have become guiding principles in my endeavors in software development, and they help me make decisions and structure my work. I am always open to improve and learn from others, so… what are your guiding principles?