Engineering accounting is a prerequisite to predictability in the same way standard accounting is a prerequisite to financial predictability. Without a structured engineering accounting system it’s not possible to accurately calculate velocity or account for the type, quantity or latency of work completed. This post describes why an engineering accounting system is essential to measuring and increasing velocity. It then illustrates the need for engineering accounting by drawing on comparisons to financial accounting. Finally, it makes a recommendation on how an organization might implement engineering accounting!

Velocity??

Velocity is a measurement which is based on the amount of work performed in an interval. The interval of interest changes based on the role that is viewing the velocity. The company may be interested in quarterly velocity, a product manager may be interested in monthly velocity, and an engineering team may be interested in velocity over 2 week intervals. Velocity is a strong indicator of performance. If a team can roughly deliver the same amount of features every interval the company can make assumptions and plan on that amount of work being delivered. It enables teams to forecast and make future commitments based on past performance.

Velocity does not ensure that teams are working on the correct tasks; velocity is pure measurement. Because it is pure measurement, it requires an accounting system to calculate. But teams can’t just measure velocity, teams also need data to help debug when they don’t meet their velocity commitments. This data also helps teams to debug their current performance and increase their velocity. How many teams can answer the following:

• What’s our current velocity?
• How is velocity calculated?
• Is velocity increasing?
• Is velocity decreasing?
• How far off from our commitments are we?
• How far off is any given team?
• What is the breakdown of a teams work?
• How do we increase velocity?
• What are the largest barriers to achieving our desired velocity?

Increasing velocity requires understanding the current velocity and where teams are spending their time, what type of work they are performing and the blockers to that work. Accurately measuring velocity requires a system that allows for objective and empirical measurement of work. I like to call this system an engineering accounting system.

Engineering Accounting Systems

An engineering accounting system is system to document the work being performed. Some sort of accounting is required to calculate velocity (the amount delivered in an interval). To understand why engineering accounting is required for velocity and predictability consider financial accounting. A financial accounting system:

Allows a business to keep track of all types of financial transactions, including purchases (expenses), sales (invoices and income), liabilities (funding, accounts payable), etc. and is capable of generating comprehensive statistical reports that provide management or interested parties with a clear set of data to aid in the decision-making process.

Pretend your company wants to understand each team's finances for IPO readiness and financial predictability. The goal is to understand each teams expenses to get an idea of the current finances across the entire org. The first team the company consults stores their numbers in a spreadsheet in euros. The second team has their numbers in quickbooks and uses single entry system in USD. Another team doesn’t use a computer based accounting system at all, but keeps physical copies of their receipts. While none of these teams are “wrong” -- especially if they are meeting their commitments and keeping expenses under control -- the diversity of accounting systems provides a significant barrier in gaining a company-wide view of current expenses. These fractured standards could jeopardize the goal of financial predictability through the overhead required to account for each teams expenses individually.

A lack of uniformed engineering accounting makes it difficult or impossible to calculate velocity and to analyze the reasons for the current velocity. Additionally, a lack of accounting system creates an environment where teams can be successful based on perception, which is subject to heuristics and cognitive biases. Good public relations, or one impactful project, may give the perception that a team is high performing, when in actuality they aren’t. A popular tool that a team produces that is frequently used could contribute to availability bias, or thinking the team that produced the tool is effective or delivers with high velocity. But in actuality the team may be ineffective or spending their time fixing bugs or in technical debt. An accounting system provides the data to make these determinations.

Accounting systems are essential to measure velocity, and a shared accounting system is essential to measure org-wide velocity. An org-wide accounting system provides:

• Structured approach in how, when and where to track work
• Shared terminology
• Shared tooling
• Standardized reports
• Lower barrier to entry when analyzing performance across teams and domains

A proper engineering accounting system can calculate much more than velocity. It’s able to answer:

• How much work is a team performing in an interval?
• How does that compare to prior intervals? (speeding up? slowing down?)
• How long doe it take to perform a task of a certain size, on average?
• How long does it take to perform a task of a certain size for a given project?
• How long does work spend in Peer Review? what percentage of total work is that accountable for?
• How long does it take to deploy?
• What’s the breakdown of work during a given interval in terms of: bug fixes, features, technical debt?

These questions are essential to understand engineering performance, and to create an environment for predictable execution and accountability (is a team making good on their commitments).

Scrum As An Accounting System

I’ve used scrum for 4 years, 2 as a member of a team and 2 as leading a team by driving the scrum process. I have found scrum just works. But in order to work, scrum needs buy-in from an entire team. A partial application (some team members using it while others don’t) render aggregations misleading or worthless. A structured engineering accounting system, like scrum, is able to answer all the performance questions outlined in the section above scoped to a single team. It offers many metrics which allow teams to achieve our goals of predictable and increasing engineering velocity, and allow teams to understand and debug the issues that keep them from achieving our desired performance.

Velocity

Scrum measures velocity by assigning each task a weight, referred to as “points” in scrum. Each work item is given an estimate which indicates it weight relative to other tasks. Given 2 tasks, each with estimate of X means each task is roughly the same weight. After tasks are assigned weights, velocity becomes trivial to calculate:

Velocity = sum of points / interval

If a team completes 20 points in a standard two week interval, that indicates a velocity of 20. If a team completes 15 points in the next interval that means there was a velocity of 15.

Note: I have found that scrum velocity is scoped to a team, and it can’t really be used to compare between teams, or as a global performance metric. Even with this limitation, I still think there is value in team-scoped velocity and an evolution on what we currently have. When a team commits to something and continually can’t reach that commitment that can be a good indicator. Velocity across teams is sort of meaningless, one team has a velocity of X, another Y, what does that even mean? How can that really mean anything to someone outside of the team? The derivative or trend of velocity becomes a valuable indicator.

Forecasting

Measuring is good, but predicting future outcomes is better! One powerful aspect of scrum is the ability to forecast velocity. Forecasting uses the data generated by previous intervals to estimate a reasonable target for future intervals. Consider a team that delivered the following points over the last 3 intervals:

15, 18, 12

The running average for this is 15 points ((15 + 18 + 12) / 3)). This enables the team to shoot for ~15 points for the next sprint. By having an idea of how much work a team can complete (their actual velocity) they are able to increase predictability. The same goes for understanding velocity when a team member is out. If there are 3 members on the team, and one member will be out for 2 weeks scrum estimates that the team should shoot for 10 points in a sprint instead of 15.

It’s really this simple, and it really works!

Profiling

Profiling shows a breakdown of work at a given point in time. It is a property of structured record keeping, and not specifically of scrum. Profiling enables teams to breakdown work by multiple dimensions, and is usually implemented through a tagging scheme. Work can be tagged by component (such as a service name, or library) or work type (bug, tech debt, feature), or any other dimension they find valuable. This enables breaking down the total amount of work by those dimensions. The charts below show an interval broken down by work type. You can see that ~40% of work is being spent fixing bugs (re-work):

This accounting enables tracking where teams are spending their time:

• Is a team spending most of its time fixing bugs?
• Developing features?
• Addressing tech debt?
• Which Projects?

Scrum provides a planning structure that lends really well to supporting this sort of record keeping.

Queueing Analysis

Queuing Analysis shows where work items spend their time. A scrum board is split into “swimlanes”. Swimlanes can be used to mirror important states in development. Some examples of these are:

• Not started
• In development
• Pull request
• Staging testing
• Production Deploy
• QA
• Done

If teams commit to accurately transitioning work items, they can get metrics on how long items spend in a given state, which is able to inform teams of where work is queueing. Once they empirically understand where work is queueing they have a foundation for structured analysis to model how changes will affect engineering throughput!!

Each of the categories above can help teams understand engineering performance, measure velocity and increase predictability.

Conclusion

I believe an accounting system is required to measure velocity. It provides minimal friction but enables teams to begin accounting for work at the team level and measuring our velocity, as well as surface the major barriers to achieving our velocity goals. I would love to hear your experiences with scrum or other other engineering accounting systems which enable the metrics described above!

Thank you!