Navigating the pitfalls of data projects
November 2024 (186 Words, 2 Minutes)
This note is based on an article from the Seattle Data Guy’s newsletter, a publication about data engineering and data project management. I keep the steps defined in the article and add my own remarks.
It highlights several good practices to apply from the start to the completion of any data project:
1) Start with the end in mind
The first step is to perform backward planning: think about the desired outcomes first, and work backwards from there. Define goals, activities, effort required, and key dependencies.
You don’t want to fall into the “technical success, business failure” trap by creating a technically perfect model disaligned with business stakes.
2) Keep things moving
Someone, usually the project owner, should ensure that progress continues at a reasonable pace. When blockers arise:
- Help teams prioritize
- Facilitate meetings between stakeholders to unblock situations
- Set clear deadlines with clear expectations for task completion
3) Define “Clear Done”
Each task or feature must have a clearly defined “done” state, a point after which it should no longer be modified or reopened. This ensures stability and prevents scope creep once a deliverable is completed.
4) Have clear landmarks
Keep the team engaged and motivated by setting clear milestones. These provide visibility into progress and create opportunities to celebrate achievements along the way.
Landmarks can also include early wins or prototypes shown to stakeholders, this helps maintain buy-in during long-running projects.
5) Publicize your project
Communicate about your project both internally and externally to raise its visibility and encourage engagement. Use emails or internal social platforms to share updates and progress.
Publicizing progress not only raises visibility but can also surface new stakeholders, users, or use cases that were not initially identified.
6) Clearly define the end of the project
Clearly defining project completion is essential. Without this, team members may disengage once the project feels “almost done” (e.g. at 98%). Establish a shared understanding of what constitutes project completion to maintain commitment through the final steps.
You can also plan for a final phase of structured knowledge transfer and maintenance handover, this helps ensure long-term impact and avoids the “project abandoned after delivery” risk.