Reading Time: 7 minutes

7 MINUTE READ

Automated Product Backlog Cleaning

Matt

Matt is the Product Architect for Enrich, our K-12 software suite for districts and states. He’s been involved with the design, development, and operations of Enrich since 2003, when it begun its transition from a being custom-developed system used by a single school district to a resalable product suited for use nationwide. As much as possible he runs, kayaks, hikes, camps, and travels with his wife. He also dabbles in music composition and plays guitar now and then.

With any software, there will always be people who request features to meet their specific needs. There isn’t enough time or manpower to tackle all of these requests at once, and sometimes they are so specific that there isn’t enough benefit to justify the change. Product managers typically are the gatekeepers for these requests, and anything that isn’t being actively worked on gets added to the product backlog. Managing this can be complicated and involve a significant amount of time.

At VC3, we strive to keep improving our processes to be as efficient as possible. So when we ran into the problem of an overfull product backlog, we devised a new approach to keep it clean and useful.

The Motivation to Automate Product Backlog Cleaning

Scrum experts advise teams to spend five percent of their time managing their backlog – so for each full-time, dedicated team member, that’s two hours per week to spend prioritizing and cleaning.

Prioritizing all backlog items regularly, say at the start of each a development iteration, creates a productivity problem: time is spent putting all work into priority order even though that order has no consequence as of yet. The further down the list of priorities you go, the more likely the item’s prioritization will change by the time the work on that item actually can start. If your situation will allow for it, a more efficient way to manage the backlog is to answer the question “what can be started next?” instead of “when should every item start?”.

This mindset is embodied in the the Lean and Kanban principles the Academic Products Team adheres to. With this approach there is less of a need to review the entire backlog which saves us time. However, some items languish in our backlog which creates clutter and wastes time when reviewing or searching the backlog. The extra clutter creates noise that is hard to sift through and makes the whole backlog less useful. So, we’re motivated to both keep the backlog clean and to minimize the time spent cleaning it.

Keeping The Backlog Clean

In 37signals’ book Getting Real, they advocate an aggressive approach for tracking feature requests – don’t track them (see: Forget Feature Requests):

“Yup, read [feature requests], throw them away, and forget them. It sounds blasphemous but the ones that are important will keep bubbling up anyway. Those are the only ones you need to remember. Those are the truly essential ones. Don’t worry about tracking and saving each request that comes in. Let your customers be your memory. If it’s really worth remembering, they’ll remind you until you can’t forget.”

The 37signals approach is a good one for some teams, and it will definitely keep the backlog clean. But, it may not work well for larger or more conservative teams where you need some way to document an emerging feature or issue.

Others have suggested an idea funnel which moves work through approval stages, but this approach still has the issue that items can languish and clutter the backlog.

Generational Backlog Grooming

The Academic Products Team put our heads together and devised an backlog aging algorithm we’re experimenting with. I’ve dubbed it: Generational Backlog Grooming. It’s similar to the way automatic application memory garbage collection works which you may be familiar with if you’re a developer.

The Generational Backlog Grooming process is straightforward:

  1. There are several “generations” (groups) of items in the backlog
  2. Periodically, items that have not been modified in the last X days are moved to an older generation. For example, Generation 1 items move to Generation 2, Generation 2 items move to Generation 3.
  3. Items that have been modified move to, or stay, in Generation 1
  4. Items that can’t be moved to a older generation are deleted (or archived if you just can’t let the items go).

Nice things about the automated backlog cleaning process:

  • It’s simple and can be automated. No special judgement from a product owner is needed. Their time can be spent doing higher value tasks like product design, product requirements, or talking to customers.
  • It relies on the entire team to modify or comment on items in the backlog to keep them alive. In other words, the collective knowledge of the team is used to age items. This could be bad though if your team is unwilling to update items or its time consuming to update items.
  • Generations make the aging process very visible to the team. This is aligned with the Kanban principle of process visualization: each generation can be a lane on the board!

Speaking of Kanban, Academic Products Team uses LeanKit’s Kanban software to manage our backlog and assigned work. Here’s what the generational backlog concept looks like in LeanKit Kanban, but it could work just as well using other Kanban boards and work item tracking systems:

example Kanban board

We are starting with three generations and a 30 day aging window but I expect we’ll adjust those as we learn more.

We have two additional lanes in the backlog. “Top Priority” which has our Next Queue cards to work on. It intentionally has a WIP limit to force prioritization decisions, otherwise everything would be Top Priority.

The other lane is “Aged Cases” which are cards that we simply can’t discard because we may need to communicate to a user that we are not doing that work for whatever reason. The discussion and analysis of these cards could possibly result in a card being moved to Generation 1, Top Priority, or Archive > Aged Out. This is an improvement over our previous backlog process where the card would have been very hard to see.

A JavaScript/NodeJS implementation of Automated Backlog Cleaning for LeanKit.

I promised that I’d post the automation we’ve written to keep our LeanKit Kanban board clean that uses LeanKit’s API. As you can imagine this topic is pretty technical!

The utility is implemented as a NodeJS JavaScript module. Like all of VC3’s open source projects, you can find this one on GitHub at https://github.com/vc3/LeanKit-BoardRules.

If you too are using LeanKit this utility may be useful to you! The utility we’ve created has nothing specific to VC3’s LeanKit boards in it which will make it easier to be adapted to your needs. Instead, the settings and rules for when cards should move around on the board are stored in a configuration file.

Let me walk you through how the configuration works. Let’s take another look at the example Kanban board:

example Kanban board

The arrows represent how cards should automatically move between lanes. In our case, let’s say the goal is that if a card isn’t updated by someone within 30 days, then we would want to move the card to the next lane.

So, the configuration rule that would move cards from the Generation 1 lane to Generate 2 lane after 30 days of inactivity would look like this:

{
cards: { in_lane: ‘Backlog|Aging Out|Generation 1’, not_modified: {in_days: 30} },
actions: { move_to: { lane: ‘Backlog|Aging Out|Generation 2’ } }
}

The rule is in two parts:

1) cards – a card selector/query to choose which cards to update based lane location, modification times, and tags. In this case, cards in the Generation 1 lane that have not been modified in 30 days will be selected.

2) actions – one or more actions that should be applied to the selected cards. In this case, the cards will be moved to Generation 2.

Here’s what the configuration file would look like that describes the entire process. Notice that there’s one rule, like the one above, for each arrow on the diagram (6 total):

// How long to wait until a card is demoted in the backlog

var demotionTime = { in_days: 30 };
exports.rules = [

// Promote cards with activity (THE GREEN ARROWS)
{
cards: { in_lane: ‘Backlog|Aging Out|Generation 2’, modified: { after_last: ‘Move’ } },
actions: { move_to: { lane: ‘Backlog|Aging Out|Generation 1’ } }
},

{
cards: { in_lane: ‘Backlog|Aging Out|Generation 3’, modified: { after_last: ‘Move’ } },
actions: { move_to: { lane: ‘Backlog|Aging Out|Generation 1’ } }
},

// Demote cards without activity (THE RED DOWN ARROWS)
{
cards: { in_lane: ‘Backlog|Aging Out|Generation 1’, not_modified: demotionTime },
actions: { move_to: { lane: ‘Backlog|Aging Out|Generation 2’ } }
},

{
cards: { in_lane: ‘Backlog|Aging Out|Generation 2’, not_modified: demotionTime },
actions: { move_to: { lane: ‘Backlog|Aging Out|Generation 3’ } }
},

// Age out cases (THE LEFT ARROW)
{
cards: { in_lane: ‘Backlog|Aging Out|Generation 3’, not_modified: demotionTime, tagged: ‘case’ },
actions: { move_to: { lane: ‘Backlog|Aged Cases’ } }
},

// Age out non-cases (THE RIGHT ARROW)
{
cards: { in_lane: ‘Backlog|Aging Out|Generation 3’, not_modified: demotionTime, not_tagged: ‘case’ },
actions: { move_to: { lane: ‘Archive|Aged Out’ } }
}
];

You’ll notice that these rules take into account lane location, modification times, and tags. After you’ve installed NodeJS as well as this module (npm install leankit-boardrules –g) then you can run the rules from the command line:

leankit-boardrules you@your.com your-password execute yourConfigFile.js

You can schedule the rules to run automatically using your favorite task scheduler (like Windows Task Scheduler). Or you can just run the rules yourself from the command line as needed.

By the way, there’s a “report” mode that will print a report of the changes before they are actually applied (“execute” mode does the real thing). This is really handy for testing to make sure you don’t accidently move cards around unexpectedly. You’ll find instructions on the project’s GitHub page for how to define and run rules. If you’re a developer and write new card selectors or actions I’d love to see them incorporated into the project: please submit a pull request.


Learn More: