I am intentionally not using the word “management” in the title of this article for a specific reason, other than making SEO people cringe. Anyone who has been a week in the software business can give you an earful about project management risks – staffing, schedule, priorities, dependencies, tracking, the list goes on. We are not going to cover any of that in this post. Anyone who is not in software business yet should check out my post on why they should be.
If you think of a project as a brand new Kawasaki Ninja 650 motorcycle, then project management would be the transmission. It is an important component keeping the engine (developers) from overheating. However, even with perfectly timed cylinders in the engine and well-oiled transmission, it is possible to high-side this bad boy, if the rider grabs a fist full of throttle while leaning in the corner. The rider would be the project leadership.
I have seen and experienced a number of ways project leadership sidetracked and buried great initiatives. The most impactful ones are carefully curated in this article in hopes you will do better than I have. I will not reference specific projects or people out of respect, all I can say – his name is Brian. 😉
1. Too many cooks in the kitchen
What does Steve Jobs have in common with the Military? They both reduced decision fatigue – one by wearing the same style of clothes every day, the other – by having a strict chain of command. When the military makes a decision, there is no ambiguity in how the process works and who makes it. If the decision-maker is unavailable it is written in the book who assumes responsibility. This allows the Army to focus on the outcome. It also makes the process of reaching the final decision quicker (yes, indeed).
Sometimes organizations get excited about a particular project and give decision making powers to a group of people. It could happen when 3 directors get together and decide to run a project or when every tech lead, people manager and architect has equal weight when making decisions. The official term for this behavior is a design by committee and bears the following traits:
- To make a decision, a consensus or majority needs to agree, which takes time. Decisions can’t be made on the spot, which increases decision fatigue.
- Decision will likely be a compromise given the unique balance of powers for every decision, which strips consistency from the entire project. You want your product to look like it was built by one person, don’t you?
There is a nice post on Smashing Magazine on why it should die. Here’s my mathematical take. Brian Whitworth has published a paper on Measuring Disagreement. Given a group of people N and the number of mutually exclusive choices K, it is possible to calculate maximum disagreement D as the average distance between opinions of any two individuals in the group. D = 1.0 means complete agreement, whereas 0.0 means complete disagreement. Let’s plot disagreement trends as a function of the group size D = f(N).
For a fixed number of options K, the larger the group size – the lower D. It means more people you add into the group, the more they will disagree. People will need to define a process and spend time going through it to reach a decision. This will slow down the project and block engineers. Not to mention, humans aren’t machines and they will forget why exactly they made certain decisions in the past and will vote differently in the future, as the number K increases.
To avoid this mistake you should designate exactly 1 individual to lead the project. That individual should be smart enough to be inclusive in the process, however, ultimately only one individual should be accountable, and consequently, empowered to make decisions. To make product feel and look like it was designed by one person, it actually needs to be designed by one person. If that individual makes a bunch of wrong decisions, then a better decision-maker should be put in his role. However, leader must be one, for efficiency and consistency’s sake.
Leader is not a title, as I have alluded to in one of my previous posts. People with more power in the organizational chain, than the leader of the project, should not feel hurt or left out. A leader is a role, just like the Product Owner is a role in Scrum. We all love our Product Owners, don’t we?
2. The rigidity of structure or process
I bet you have seen something like this in your career:
– “Hello, tech support, I have just burned my hard drive rebuilding entire repo for a customer demo tomorrow. I’d like a replacement drive please.”
– “Hi Brian, no problem. We have ordered a new hard drive for you. It will arrive in 2 weeks.”
Perhaps you have seen this situation as well:
– “Hello Brian, you are an administrator on our test AAD tenant, could you please click a button to grant our test application read-only access to graph API?”
– “Please create a ticket in my backlog and schedule a security assessment for your application. You are currently #8 in my priority list so I should be able to look into it in about 3 months”
Here’s another dialog that takes place when the deadline is looming:
– “Hey Brian, could you fix that bug in our replication layer while Jason is out of office since you have worked in this stack for 5 years?”
– “Sorry, you have to talk to my manager who is currently traveling and will be back in the office on Monday.”
I can keep going but I think you catch my drift. Disclaimer – all dialogs are completely fictional and any resemblance of real-world situations is purely coincidental.
It is a mistake to approach project execution as an SOA – organization provides a set of services to the project and the project must consume them in a well-defined interface in priority order like every other project in the organization. Most common manifestations of the problem at:
- At the individual level
- Work on the tasks strictly in the order of the backlog.
- Use a bug tracking tool to have a discussion, like JIRA.
- Refuse to learn new technology and wait for the next task in the area of expertise.
- At the team level
- Use a regular weekly meeting between project leadership and a specific team to discuss urgent issues.
- Use a planning process that aligns with the rest of the business rather than rhythms of the project.
- At the organization level
- Force project discussions to happen up and down the management chain and stifle lateral collaboration (also known as building a castle).
- Require project to have a higher priority in the organization before providing any help, including face to face guidance, explanation or a coffee chat.
- Refuse to give exceptions to any rules and policies whatsoever.
Teams in the business operate vertically – meaning, they are staffed to perform business functions in its entirety and most decisions can be made strictly up the management chain. Projects, on the other hand, tend to operate horizontally spanning multiple teams. As such, they are prone to more friction. For example, to release a new edition of SQL Azure database, you have to bring together:
- Control Plane team to work on API, workflows, and CLI.
- Portal team to build UI.
- Backup/Restore team to calibrate restore plans and certify point-in-time restore SLA.
- Resource Governance team to work on CPU, memory, IO allocation and availability of the instance in the cluster.
- Geo DR team for certification of UPDATE SLO workflow and ability to see across database editions.
- Storage engine team to consult about pit-falls of higher page file sizes.
- Capacity team to work on capacity model adjustments given estimated customer adoption.
If the Control Plane team decides to live their own life, API structure may not match what the Backup/Restore team has built in the workflow. When user clicks a button to copy his database in the portal, it may end-up deleting it instead. If the Resource Governance team refuses to play ball, it may miss extra CPU requirements during the full backup phase, which will starve customer workload. If Capacity team goes AWOL… well, I guess no one will notice, really. Capacity is a constant crunch in the cloud business – who cares if you light up a few trees if the entire mountain is on fire. 😊
The Wrecking ball
Leadership must manage the project like innovation – with urgency and calculated risk. Project leader needs to convince the stakeholders that they have skin in the game. As strange is it may sound – people forget that projects exist to advance the business. A rising tide lifts all the boats, as they say. If the initiative fails, the business doesn’t just return to the status quo – it actually falls behind the competition. When enough projects fail, it renders the company irrelevant in the market. Customers won’t do business with the company that can’t get it’s act together and is stuck in 1990.
Project is a war zone so peacetime laws do not apply. Rather than being a “judge”, management should act as “co-defendant” and ask “how can I help today?” every day. It is leader’s job to identify all “castles” in the project, break down barrier walls to build bridges of communication and collaboration. Things that take weeks to accomplish when it is business as usual, should be completed in a matter of days. People need to talk to each other regardless of how far apart they are organizationally. People who can help with guidance or information should do so, even if they don’t have time to actually sit down and code. There is no path leading to a manager being rewarded for preserving his or her castle on the ashes of a cross-team project. It is the project leader’s job to shake every wall in the project until there’s nothing left. And on the ruins of walls, leadership must erect bridges.
You have been reading a lot of text and deserve a picture. These two are copyrighted, please do not share.
We all enjoy the flawless performance of a chamber orchestra – every sound is intentional and well thought through, timing is perfect, the volume of every voice is right to elicit an emotional response. Now imagine a flute player bringing an electric guitar instead, or a drummer with a left hand faster than the right one. Suddenly performance is not as enjoyable, is it?
A similar thing happens to the product if not enough details get not enough attention. Quoting one of my former organizational leaders – “Garbage in – garbage out”, meaning if engineers do not question every design choice and every line of code to the fullest extent, they will produce less than ideal product. He liked to give an example of people, who obsess about the backside of the cabinet – meaning, that no one can see the backside of the cabinet when it is standing next to the wall. Despite the backside being invisible, it is still integral to the overall product and those people who obsess about the backside, end-up making a superior product than those who ignore it.
Modern electronics is so thin because every team working on PCB optimizes their parts in concert – team working on a camera assembly would talk to the battery team to purchase the right parts to avoid overlap to reduce device thickness by a millimeter. Rarely electronic consumers pop open their devices (analogy to the backside of the cabinet), yet everything inside has been obsessed over by a large number of people.
If the same principles are applied to software business, the instinctive reaction of the team is to push back on micro-management. Every engineer and team lead wants to enjoy the freedom to do his or her thing by his or her own vision. After all, it is important to empower people and give them the runway to grow, right? The traditional contract stipulates than one shall convey the requirements to the team and return no earlier than 2 weeks to see what they put together. This process makes pretty bold assumptions:
- Whoever writes the requirements can express them with enough detail right off the bat.
- Whoever reads the requirements actually understand the “why” behind every line of text.
Hold your horses my agile friends, I know what Product Owner does.
In reality, humans don’t know everything and the larger the project, the less they know upfront. As time progresses, humans discover new things and need to keep adjusting to initial assumptions and requirements. This is called cone of uncertainty.
Needless to say, sometimes it is not even obvious that certain requirements exist until you talk to two teams and discover the disconnect. For example, a database tier team needs a Spinnaker pipeline, which in turn needs a Kubernetes cluster, which is bootstrapped by a team that needs a database to store state.
The Control freak
Project leader’s job is twofold:
- Be that one individual who obsesses about every tiny detail of the product. By doing so project leader establishes a quality bar for others to follow. He serves as the last line of defense if no other two individuals talk on the team.
- Inspire other technical leaders and engineers to obsess about product details. The intent is to set an example and give the tools to engineers to poke each other. When engineers start having passionate conversations during code reviews – it is a sign of success.
Striving for product perfection will definitely earn you a micro-manager label. It will take time to make people see the value in the depth at which project leader operates in every part of the project. Perseverance is key. People will push back but will come around and join the forces once you catch a few nasty design choices.
Initially, people will see you as the one blocking forward progress and they will be right. That’s why it is important to strike a balance between endless design discussions and letting go of some imperfection for the sake of moving forward.
Here’s my methodology:
- Ask people to write design documents. If people start coding without writing a single page of text or thinking for 15 minutes – no go. Have people spend a day laying out what the will do and why and educating their team members and partners.
- Look for design assumptions in code reviews. Project lead needs to be on all code reviews for the project, at least initially. Sometimes you can catch a benign code change, which blocks IOCP thread for a few seconds. Discuss design assumptions with the engineer and if there’s a disagreement – ask for a design document.
- Help with implementation. “It is just one memory allocation” (of 150KB per request). Use this opportunity to elevate the engineer’s awareness. Perhaps he is new to CLR and doesn’t know the threshold beyond which objects go to LOH and what it means for CPU utilization on the server with high throughput. After all, engineers need to come out stronger after project completion.
- Join stand-ups and ask for progress updates. You need to have insight into how fast each individual is moving on the project. This will allow you to focus your attention on stragglers.
- Help stragglers. Walk around the office and ask if people need any help with the project. Sit down and code with people.
- Use the product. Be the first end-to-end tester of the product. Find problems and let teams know. After all, you have “micro-managed” the team so what they produced is your fault. Allocate regular time to explore the product you are building to see if it matches what you think you are building.
- Code. A hands-off leader is useless. As a project leader you have to have a piece of code in the product written by you. It is best if it is a foundational piece of code, but even a UI button will also do.
- Suffer. Not trying to sound too dramatic, but no CI/CD pipeline is perfect. Make sure you go through the same “pain” your teams are going through. You will see inhibitors that teams aren’t talking about because they got used to this mess.
- Take accountability. Given your level of involvement at this point, you should stick your neck far and high for anyone to see. If something goes south with the project – it is your fault. If your product is mining bitcoin on customer machine – that’s your fault too.
Over time, as cone of uncertainty shrinks and engineering horizons expand, it is appropriate to remove yourself from certain depth in the project and let people run independently, purely based on trust.
Sometimes engineers act like people and need the motivation to keep moving forward.
A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools.Douglas Adams
Everyone is familiar with a Hype Curve by Gartner. I’d like to redraw it specifically for a project life cycle.
The highest point on the chart is reached when everyone on the team understands what they need to build but haven’t started doing the work yet. From here hype can only go down and it is up to you to manage the slope. There are only two things you can do:
- Push the lowest point further out in time.
- Push back on the rate of descent (e.g. increase the altitude of the lowest point).
Engineers lose excitement when they are blocked or confused. The longer they stay in that state the deeper their excitement drops. Their productivity is a monotonic function of excitement which means if they lose their muse their productive output would drop as well.
- They are physically blocked. They are waiting for someone (you, from paragraph 3?) to approved/sign off on their design, or they are waiting for a new keyboard because they passionately destroyed the last one.
- Lack of clarity. Engineers can’t make a decision – either too many options or they don’t know which option is the right one.
- Lack of tools. Engineer don’t know that Find-Replace functionality exists in their IDE hence they manually replace every occurrence of string “foo” with “bar” in the text. Alternatively, they can also get stuck because tools haven’t been written by dependent teams yet. The most common situation here is that tools aren’t up to snuff at the time when they are needed. By the time tools mature, engineers are already disillusioned.
- Lack of direction. The project keeps changing desired outcomes every week and they would rather do nothing than scratch and redo what they have done last week.
- Lack of support. If engineers smell they are fighting an up-hill battle and no one really cares – they stop caring too.
- Lack of appreciation. We are tapping into people aspects here a bit, but that’s OK. Engineers want to be appreciated for what they do.
- Technology shortcomings. That rocket booster you thought you were building, can only lift a grocery bag 10 feet off the ground. There will always be a gap between what you thought technology was going to do and what it actually does.
Simon is a motivational speaker and for your project to be successful, you should become one too. One of the most important tools of a project leader is communication. You just can’t over-communicate so try your best.
Have a direct line of communication with every engineer on your project to share information and gather feedback. Communicate in email, slack channels, in-person, in group settings. Err on the side of broader communication over filtered or segmented. Through communication, you will bring clarity, direction, and partial support and appreciation. It even helps to act as a network hub – any shred of relevant information you receive just broadcast to the entire network.
Target individuals with different incentive levers. I touched on incentives a bit in this post. If someone is motivated by challenges – create a competitive environment and reward top performers. To be successful in motivating people you need to know what they want – spend time to get to know your team individually face-to-face.
Minimize time when people are blocked on tangible things – you, the tools or the process. If people are constantly blocked on you, clearly you are overextending and need to pull back. Rally your partners and managers to help with the process and work with your TPM to get the tools ready in time for the development season.
When it comes to technology not living up to the expectations – minimize the time the team spends in this phase and expedite getting out of the pit. One trick you can use is don’t make everyone go through this phase. Instead, designate your trusted team of SEALs who will attack the lowest hanging fruits before general wave of developers reaches the first integration point.
5. The right people for the wrong job
Just like a manager carefully selects people for his team, a project leader needs to select people for the project.
– “What’s your biggest weakness?”
– “I don’t know when to quit.”
– “You’re hired.”
– “I quit.”
(Credits for the joke go to ZoomInfo.)
Initial project staffing is proposed by management unless you happen to work for a company that signs a blank check and you can have anyone. In reality, your initial team competence will resemble the bell curve – a few really good engineers, a good chunk of people that will get things done but will need some help. A the tail end of the curve a surprise awaits you. You may find salespeople there (Put your pitchfork back in the barn – nothing wrong with salespeople, this is engineering post), people on vacation or extended leave, or people who just left the team or the company. Occasionally, you find people who were put there by their managers to push them for a promotion – semi-political move, so to speak. My favorite type of people to find there – those to be hired. Yep, I have seen managers putting future hires on the project happening today. 😎
Your job is to meet project staff, get to know them and size them for the opportunity. Do not trust project roster until you’ve accounted for every soul on it. When you meet individuals, evaluate where they fit it and how you can serve them.
Speaking of serving, I recommend reading Serve to Lead book by James Strock. Every other word in the book rubbed me the wrong way, but I still recommend it because I agree with the point the author makes.
The outcome of your round of introductions should be feedback to management about staffing quality. You may ask for more people or replace some people with others. Once the roster is finalized, you have to have a clear idea of how you will help the team execute and grow. The project will not be successful without people so you have to bring people with you on the journey.
When the project kicks into gear, you need to maintain the feedback loop back to individuals on the team and their managers about their performance. Timely feedback with the best intentions will build trust and produce the right effect.