The February 2026 Tech Talk was presented by Dr. Bill Nichols and Dr. Tapajit Dey
Abstract
What is a credible range of dates for when the work in a product increment might finish? Software development work includes many sources of uncertainty, including the size of the work product, developer work effort, complexity, and waits that are sometimes predictable. We found that story point counts often don’t add information that is useful for projecting durations to completion, and wanted to understand whether the problem was in estimation or execution. To investigate, we compared data to determine the correlation between estimates of stories, the actual story direct time, and the calendar durations for story completion, in the hope that this would help to better predict credible completion ranges.
Our research analyzed real data (TSP) from 39 enterprise software projects, including 7,400+ work items. We used the TSP data, with detailed accruals and plans, as ground truth and compared with that same data synthesized into Jira-like data set (lossy compression). We proceeded to quantify sources of variability and bias. We found the following:
- Real stories often spend significant time in wait states that appear only indirectly through measures such as work in progress.
- Story points capture an estimate for the effort that a piece of work might require, but they don’t correlate with the actual effort, nor do they predict the actual duration of stories. Without story level measures, there is no meaningful feedback which may explain the persistently weak correlations.
- Cycle time (the calendar duration it takes to complete a work item) fails to adequately measure effort, due to embedded wait time from task switching and nonwork time.
- Ironically, wait states created the most noise on the shortest stories. Shorter sprints are becoming an industry standard, but they may lead to larger swings in velocity when work items are short or straddle sprint boundaries.
This talk presents a straightforward, data-driven approach to understanding these effects and improving measurements; and how both estimation and measurement can include these effects to make better plans and better projections for commitments.
About the Presenters
Dr. Bill Nichols is a Principal Engineer in the Software Solutions Division of the Software Engineering Institute at Carnegie Mellon University, where he leads the Software Measurement and Analysis team. Before joining the SEI, Dr. Nichols earned a doctorate in physics from Carnegie Mellon University, after completing graduate work in particle physics. He later led a software development team at the Bettis Laboratory near Pittsburgh, Pennsylvania, where he developed and maintained nuclear engineering and scientific software for 14 years. He has more than 30 years of technical and management experience in the software engineering industry and has published in Nuclear Instruments and Methods, IEEE Transactions on Nuclear Science, IEEE Computer, and Physical Review Letters. He is currently the Principal Researcher for the Automated Continuous Estimation of DevSecOps Pipelines project that measures software processes for project and program management.
Dr. Tapajit Dey is a Member of Technical Staff (MTS) Researcher at the Software Engineering Institute, Carnegie Mellon University, where he works in the Software Solutions Division. His research focuses on empirical software engineering, AI-augmented software engineering, and mining software repositories. He earned his Ph.D. in computer science from the University of Tennessee, Knoxville. Prior to joining SEI, he was a postdoctoral researcher and later a research fellow at Lero, the Science Foundation Ireland Research Centre for Software at the University of Limerick, where he worked on inner-source and open-source software development and helped to found the Lero Open Source Program Office.
