Metrics That Matter

Zoom webinar to over 24 CEOs and engineering leaders

Last week I was delighted to have the opportunity to share some of my engineering leadership experiences with over 24 CEOs and their engineering leaders. I presented an online webinar on the topic of “Metrics That Matter” hosted by private equity firm Pacific Lake Partners which invests in search fund entrepreneurs and their journey to acquire and build a technology company. A special thanks to Scott Barstow, on the Value Creation Team for reaching out and asking me to present to their group of companies and also helping with some of the slides.

The topic of Metrics that Matter is popular among CEOs and their engineering leaders because all the companies that they are leading have big initiatives for accelerating growth. After all, how do you improve what you can’t measure? In other words, if we start to make organizational changes how do we know those changes are moving the needle in the right direction and supporting overall business outcomes? I shared my perspectives as a leader who’s led engineering teams through 5 M&As totalling over 55MM in investments.

Key Takeaway – From an engineering perspective, what KPIs should we track and how do we ensure they support the overall company objectives? In the following diagram I shared during the webinar, I use the management methodology known as OKRs. However, the same could be drawn for similar frameworks like Management by Objectives (MBO), or the Balanced Scorecard. I recommend reading Measure What Matters by John Doerr, which is a great book to learn about OKRs.

Your KPIs should align with Key Results or should be considered for removal

Summary of Diagram – At the top of the diagram, every company should have a big audacious goal set out with a long term horizon (usually 3-5 years). This mission, or north star, establishes the overall direction and singular focus for the company. The next two rows, represent the OKR process. This strategic management process helps align individual and team goals with overall company objectives, while providing visibility and accountability for everyone in the organization. Typically this planning happens annually with objectives broken down into quarterly and monthly meetings to measure key result outcomes. The bottom row represents the teams KPIs which are a set of metrics tracked over time that gives the team an overview of their performance. These KPIs are often tracked weekly, bi-weekly, or monthly depending on the team and their preferred cadence. I won’t go into the process details at each level as there are lots of courses, books, and blogs you can read on those topics.

My Recommendations:

  • Make sure that your KPIs “show up” and inform a business Objective via a Key Result on a regular basis or should be considered for removal. Sometimes a KPI will not inform and be tied to a Key Result, but that should be the exception not the norm. If there are KPIs that the team feels will help them keep tabs on their own performance and help identify actionable insights, they should be tracked internally (inside the team or area of competence only) and not part of this overall company process and reporting.
  • Make it comprehensive – I’ve seen instances where the company only performs OKRs and tracks KPIs but lacks the top level (mission/north star) which helps everyone in the company, including individual contributors, understand the big picture and their purpose. Ever hear of the story of the three bricklayers and the parable about the power of purpose?
  • Educate everyone in the company – This process is not just for managers/directors and up. It should be taught company wide and part of the new employee on-boarding schedule. It will help new and existing developers, who are working on a feature or fixing a bug (“the what”), understand the bigger purpose of what they are doing (“the why”).
  • Evolutionary not one-and-done – Every company will be different and every product will have its own set of KPIs that eventually tie into a key result to support an objective. What I’ve found works is try and measure some well known KPIs and test it for a few months to make sure it provides value. Do a retrospective on it . If it can’t pass the scrutiny of the team and support a key result, either remove it. Or keep it internal to the team only as mentioned in the first point. Over time, you should end up with a tight set of KPIs that are meaningful and provide the best value to support company objectives, and overall the mission of the company. And evolve them continuously as the business evolves.

Some examples to try:

Look up these KPIs to find out more about them, I’ve included the names but left out their descriptions and purpose for brevity of this writeup.

  • Code coverage
  • Bugs outstanding (sev1,2,3)
  • Bugs fixed vs reported
  • Deployments per week/month
  • Points delivered vs committed (or Value delivered – going beyond velocity)
  • Cycle time (vs lead time)
  • Escaped defects
  • Uptime
  • Response times
  • Hosting costs/client
  • Team happiness
  • Pull requests (average time to merge)
  • … likely a lot more but try these to see how they fit in your organization if you’re not already using them!

KISS – Remember to keep the list of KPIs as small and as simple as you can. Through team scrutinization (retrospectives), remove the ones that are not providing value as the business changes. Automate tracking of these KPIs whenever you can. If your team leads/managers are spending more than 1-2 hours a month gathering KPIs then that’s probably not as valuable anymore (the law of diminishing returns).

Thanks again to all the CEOs and engineering leaders from the Pacific Lake companies that set aside over an hour of their time to listen and challenge me with questions to help their company!

Fastest Growing Company in North America List

This just in, we’re ranked 367th on Deloitte’s Technology Fast 500™, a ranking of the 500 fastest-growing technology, media, telecommunications, life sciences and energy tech companies in North America. It’s been a busy year and to be on this list validates a lot of things we’re doing right at eCompliance. Being 1 of only 23 other companies in Toronto is even more special because the city is a growing epicenter of tech companies in Canada. Toronto even competes with prominent cities like San Francisco and Seattle in the US in terms of total number of tech jobs and growing tech markets!

Earlier this summer a significant $40 million investment came from Alcumus Holdings Ltd. This marks five M&A’s and one angel investment (totaling over $55 million) that I’ve been a part of as head of software development (11 years). With a little fun and to recognize that you learn new things everyday… here’s a picture of my Deal Toy, used to commemorate the closing of a business deal!

Luu Duong Deal Toy 2019

closeup deal toy

I’ve never heard of a “deal toy” before getting one of these, apologies to all the investment bankers, private equity associates, and venture capitalists out there that probably have a dozen of these in their offices!

Wonder what 2020 will bring for new challenges and professional growth? Hope it’s even bigger than 2019!

How walking skeletons eating a slice of cake can help with collaboration?

Walking Skeletons Eating a Slice of Cake

Introduction:
Collaboration magic is still missing from many agile team’s processes. Traditionally, developers worked independently and often didn’t have high collaboration with the business, QA’s, UI/UX, or with external customers. They treated those roles as “phases” that happened before and after development in a step-by-step phased approach. As a developer if you’ve worked in a low-collaboration environment that practiced phased-gate development (business requirements-> design-> development-> testing-> release/maintenance) you’ll find it very frustrating that there is a lot of back and forth towards the end of the cycle… the last mile takes so long. With a newer agile approach, we’re supposed to work in a much more collaborative environment but sometimes old habits die hard and actually prevent collaboration. How we “used to break up the work” remains the same and conflicts with this collaborative magic. Here’s a suggestion when breaking up your work to allow for high collaboration to happen.

When working on a user story try to divide the work up into pieces of a cake, a “slice of the cake with all the layers present”. If some layers are complex you can even mock them out (fake it) assuming they are implemented for the sake of getting early feedback. Getting feedback early and often (and doing desk checks) is how we can work on an agile team with many contributors collaborating to finish the story and deliver value to the customer.

A “piece of cake” is a vertical slice of the story – it includes all component layers like database, repository, domain (business layer), controllers/view, with front-end UI/HTML/CSS if you’re on the web. See diagram below that describes a typical epic story (cake) and how to deliver it via fully functional parts (slices of a cake) with all the component layers. Breaking up a story (the cake) into vertical slices instead of horizontal slices (the components) allows you to get quick feedback from your team as well as customers.

vertically slicing user stories

How to approach it:

  • Concentrate only on the most important business value first (something a customer will use 80% of the time, skip exceptions, skip things that customers only use sometimes, skip doing all aspects and filling in the rest once you get something that is a proof of concept, something you can show and get feedback on. In technology spikes this is called a “tracer bullet”, it is the quickest way to get a proof of concept.
  • Only include a few fields (the most important fields that pertain to the work at hand). For example if a form has 15 fields only implement the 3 that is most important to get the form submitted so it can go onto the next step.
  • Only include a subset of the feature (remove validations, hard code some settings, remove unimportant aspects)
  • Build only the most common workflow, or “work on the happy path scenario first”, then add other less common workflows later.
  • Mock complex layers or third party integrations in your first slice, mock it until you get around to working on it. But remember it’s good practice to learn these complexities earlier on so if you can get the feedback you need in the first slice then implement these complexities during the first 33% of the story work would be ideal.
  • Create “Walking Skeletons“, a term coined by Dr. Alistair Cockburn where you create a small implementation of the system that performs a small end-to-end function. It need not use the final architecture, but it should link together the main architectural components. From there the architecture and functionality can evolve in parallel.

Benefits of doing this:

  • Speed – Speeds up the feedback loop. Hand-offs are slow, collaboration is fast, opportunities are given to the team to adjust at smaller intervals as you work towards the final output. Rework is costly when you have a large output, but will be small as you increment those changes throughout the development process.
  • Agility – Allows us to change if we’re going in the wrong direction. Sometimes you’re going in the wrong direction and you’ll be wasting time if you don’t do desk checks early enough so you can quickly adjust based on feedback. Having something demonstrable allows you to get this feedback early if you’re on the wrong track.
  • Collaboration Magic
    • Allows QA to do desk-checks earlier. QA team members don’t want to be the bottle neck at the end, which is often the perception because when developers build layers there’s not much to test until significant layers have been 100% coded. QA’s can also start their functional automations a lot earlier as well so they’re not writing automations for the previous completed sprint, which is often the case if you have low collaboration.
    • Allows UX to do desk-checks earlier. There are good tools to mock UI’s that have user interactions built-in. But the UI layer mockups is only a start, it’s limited to all the user interaction feedback that UX professionals need to get when the software is actually running. Eg. Have you ever seen a workflow that calls to an external API that takes a little too long? You might find that adding a progress bar or some loading/waiting prompt is easier than asking the third party API provider to make changes to their infrastructure. This insight only comes as all the component layers are connected to make a “slice of the cake” and you get functionality that supports the interaction design that you need.
    • Improving team performance, via this collaboration practice, pays off exponentially compared with improving individual performance. This is a whole separate discussion topic which you can read here.
  • Product Learning –  Learning not only happens quicker within the team and the technology, it gives you opportunities to actually show customers and get valuable feedback much earlier than waiting for the entire 5-6 slices of a story to be complete (as is the case when building components). Your UX and Product team often have pilot customers or customer advocacy groups that love giving feedback and input on the things you’re working on. These early adoptors of your features or stories can help you steer the remaining stories on the right path so that you’ll have less rework if you were to show them the “entire cake” many months down the road.
  • Deliver Quicker – Allows you to deliver completed work even if the entire story is not 100% complete. Eg. If your cake was made up of 4 slices and you delivered 3 slices (75% done), it’s still valuable to a customer cause they can perform 3/4 of the story completely. Contrast this approach to delivering 3 layers (domain, repository and the database layers) that doesn’t provide value from a customer perspective (the UI that they need to interact with is missing). Even if you’re 90% done, without a UI you might as well be 0% done. In the first scenario you’ve delivered 75% of working features, in the 2nd scenario you’ve delivered nothing to a customer because they can’t use anything. Creating a “walking skeleton” is better than having a full body with no head or brain to control the rest of the body parts. Remember that delivering value to customers might be different depending on where the product is at in the technology adoption life cycle so release with that in mind!
  • Lower Risk – Technical risk as well as story complexity can be identified earlier for the story, not just at the end when you finally connect the last layer to find out that you have complexity in one of the layers. Or finding out that your most complex layer is the last layer you decided to do last and that work requires a bunch of extra time, ruining your teams estimates. The complex component is discovered quickly because it’s part of the initial slice you work on! It’s great to be able to identify risk earlier on in a sprint which helps you more accurately scope the remaining work required, making your team and Scrum Master very happy!

What to look for if you’re doing it wrong?
As a developer, how do you know you’re not doing this correctly? What are the smells to look out for if you’re doing it wrong?

  • At standup you say you’re working on the same story over and over again, but nothing is ready for desk checks from UX, QA, Product until you’re really far along.
  • Take note the scrum/kanban/agile board that you’re using, are your stories moving incrementally every day?
  • You’re half way through the story and you’ve not interacted with QA or UX or gotten much feedback from the product owner. You should actually try to get this earlier even at the 30% mark, it’s possible if you break up and work on the “slices of cake”.
  • You find that QA or UX is always giving feedback way to late in the game and the story completion is always at risk, causing you to rush last minute feedback from other team members at the end of the iteration.
  • Rework. When you do get feedback it causes you to change a lot of code or logic. This means you went too far without getting feedback, undoing code or adding a lot of logic to get the new feedback implemented is a huge sign of not following this approach. If you build it lightweight, just enough to get feedback then changes that you get from the feedback should be small and incrementally forward moving.
  • Does QA or UX or even product team members always seem like they’re the bottleneck to being able to say “we’re done” to a story? Most of the time, it’s because they were involved too late in the process!

Now you can change this!

Rule of thumb
Although this is not always possible, having the mindset in place and practicing “slicing the cake” or creating “walking skeletons” will help you work on a team and deliver better results because you enabled early feedback and collaborated with other team members. So instead of throwing the work over the wall or waiting until you’re fully complete your part of the process (limitations of the old process). In my experience 95% of the time it should be possible, so use that as rule of thumb!

Additional references…
Think of a whole story as a multi-layer cake, e.g., a network layer, a persistence layer, a logic layer, and a presentation layer. When we split a story, we’re serving up only part of that cake. We want to give the customer the essence of the whole cake, and the best way is to slice vertically through the layers. Developers often have an inclination to work on only one layer at a time (and get it “right”); but a full database layer (for example) has little value to the customer if there’s no presentation layer.

~ Bill Wake

http://www.deltamatrix.com/horizontal-and-vertical-user-stories-slicing-the-cake

Related to user stories but can also apply to working on an architecture story or a technical spike, AKA the concept of “tracer bullets” from the pragmatic programmer http://wiki.c2.com/?TracerBullets.

It’s a best practice to use the INVEST mnemonic https://en.wikipedia.org/wiki/INVEST_(mnemonic) when creating or breaking up user stories into smaller manageable parts. This practice aligns well with creating “walking skeletons” and working on “slices of a cake”.

Experiencing products through conversation

Amir Shevat, VP of Product at Slack.com presented a great perspective at Lean Startup Conference 2015. The talk was titled “Irrational Products – How to avoid bad product calls”, where Amir talks about experiencing products through a conversation. These conversations can be used by Product Managers to validate their products. His example starts off by taking someone from the crowd and asking him to come on stage. Then Amir, pretending to be the product, and the volunteer being the user, proceeds to ask questions to the user about giving up his username and password to start using the product. Amir explains how the conversation gets awkward, especially since the user didn’t get any perceived value from the product yet. But this irrational behaviour is often the conversation users have with a lot of products (at Google they call this the “wall of login”). Amir recommends having these conversations, aka “talk the app”, to make rational product calls. To see the full presentation as well as a few more recommendations from Amir on making the right product calls check out the recorded video session.

 

 

Become a better leader – Agile Coach Camp Canada in Calgary – June 12-14, 2015

I am one of the organizers for the first Western Edition of the Agile Coach Camp Canada to be held in Calgary from June 12th to 14th, 2015. Come join us as we share experiences and learn better ways of leading teams.

Let’s have a conversation about what it means to be an agile leader, why it matters, and where we are headed. The annual gathering at Agile Coach Camp creates opportunities for our coaching community to share our successes, our learning, our questions and our unresolved dilemmas – all in an energizing and supportive environment.

Agile Coach Camp Canada West is an open space un-conference inspired by the original Agile Coach Camp. Calgary, Alberta is excited to host Agile Coach Camp Canada, West Edition. This is a practitioner-run conference for peer-to-peer learning, exploration and networking for ScrumMasters, coaches, and others helping to transform organizations.

This event has amazing value at $75 for the entire weekend. This price includes all food & drink for Friday evening, Saturday (breakfast, lunch & snacks) and Sunday morning.

For more information, hotel accommodations, or to register, visit the Agile Coach Camp Canada West website at http://agilecoachcampcanada.ca/west-2015/agile-coach-camp-canada-west/­

March 5th, 2015 – Hosting Klaus Boedker at Calgary Agile Users Group

Join me, Thursday March 5th, 2015 as we host Klaus Boedker at CAMUG (Calgary Agile Methods User Group). Klaus will be presenting “From doing agile to being agile: 10+ ways of developing an agile mindset.” The goal of this talk is for you to:

  • Learn and share the benefits of being agile over doing agile
  • Increase your level of fundamental agile thinking and understanding
  • Be able to identify and share 10+ opportunities for growing an agile mindset for yourself, your team and your organization
  • Learn the brain science behind how to get people and teams out of the ‘agile autopilot’ mode

Full details here: http://www.meetup.com/Calgary-Agile-Methods-User-Group/events/220430262/

See you March 5th at Innovate Calgary 6pm!

Nov 6th, 2014 – Hosting Dave Sharrock at Calgary Agile Methods User Group

Thursday, Nov. 6th, 2014 we’re hosting Dave Sharrock at CAMUG (Calgary Agile Methods User Group). Dave will be presenting on Lean Product Management: How to effectively validate your product or idea? Join us at Innovate Calgary at 6pm as it’s going to be a full house!

Abstract:
The rise of the Lean Startup has led to a deeper understanding of the importance of validating business ideas, from new features to new business models. But many tools available to the Product Owner aren’t adapted to rapid validation. Starting from the principles and practices of agile product management, from defining the product vision to creating story maps and refining the product backlog, you will learn about key practices that incorporate the lean startup principles, allowing a Product Owner to bring the build-measure-learn cycle alive and ultimately earn more value more quickly.

Full details here: http://www.meetup.com/Calgary-Agile-Methods-User-Group/events/213451192/

Locally testing a self signed SSL certificate in IIS manager and getting “The process cannot access the file because it’s being used by another process” error

Making changes to our shopping cart for Google Analytics, Conversion, and eCommerce tracking. In order to test a local version of the shopping cart I had to make and use a self signed SSL certificate.

 

2014-11-28 21_39_57-Clipboard

Every time I try to start the site “learning2” with the binding to port 443 (https) in IIS Manager I get an error message:

You receive a “Cannot Start Web Site” error “The process cannot access the file because it is being used by another process” error message when you try to start a Web site in the Internet Information Services MMC snap-in

2014-11-28 21_41_05-Clipboard

 

Reason and Resolution: Another process is using port 80 or port 443 on the computer that is running IIS. By default, IIS uses port 80 as the default TCP port and port 443 for Secure Sockets Layer (SSL). In order find out what’s using port 80 and 443, for HTTP and SSL respectively, you would enter this at the command prompt:

netstat –ano | find “:443”

netstat –ano | find “:80”

2014-11-28 21_41_32-Clipboard

In the output see the process ID “9820” for localhost using port 443? Next start windows task manager and look at the “Processes” to find out the culprit. You might have to turn on the PID column to see it (view->select columns). For me it was Skype that showed up as the process that matched the process ID “9820”. After turning Skype off entirely, I was finally able to start the site that was bound to port 443 in IIS Manager. Hope this article helps anyone else getting stuck on starting up a site in IIS. It could be a number of other programs using the ports so the best thing to do is check using the netstat command and matching up the PID with what you have in Task Manager.

Here’s the Skype setting that was causing the issue (Tools –> Options –> Advance –> Connection) see below. You could also leave Skype running and just turn off the ‘”Use port 80 and 443 as alternatives for incoming connections”. Remember to restart Skype to propagate the settings.

2014-11-28 21_42_17-Clipboard

 

Shortening the feedback loop: phone interview

Been a while. Here’s a podcast about the book contribution the team and I worked on last year. The topic of that write-up was “Shortening the Feedback Loop” and hopefully it will be published soon. I only had 15 minutes to prepare for this interview so it kind of caught me off guard. I definitely have some room for improvement on my phone interview skills. Oh well, that’s the way things go sometimes… “you know!!”… http://www.blogtalkradio.com/rapidbooks/2011/04/04/agile-stories-retrospect