kasperhermansen-blog/content/posts/2023-12-08-getting-through-a-big-project.md

---
type: blog-post
title: Getting through a big project
description: In this article, I share how to navigate a project, identify key moments of crisis, and how to solve them.
draft: false
date: 2023-12-09
updates:
  - time: 2023-12-09
    description: first iteration
tags:
  - "#blog"
---

## Introduction

As a engineer you will work on a variety of projects, some small, some large,
some complex, and some you can handle in 15 minutes. But once in a while you
will be faced with a big project. You may not know initially that it is gonna be
a big project, but somehow it turned out to be.

In this article I am gonna share some of the experience I've accumulated,
working on big projects, the various stages, and surprises you may will face,
and that it is gonna be okay.

This article is mainly geared at new software engineers, but is applicable to
most engineers as it focuses mostly on the problem domain, not the technical
skills.

## What is a big project

I categorize a big project to be around 3 months or longer. This depends on your
own internal barometer of what you can stomach but to me, I like to do small
changes, fast, iterate and so on. 3 months and longer would be a project me and
the team would have to work on for a long time, and face a natural project
development lifecycle which I will share later.

A big project are usually constitute a few factors, each factor you add to the
project it will multiply the amount of effort required.

1. Cross team collaboration:
   1. Do you need external parties to succeed.
   2. Do you need to break some interface to another team, i.e. does a design
      change, which requires work from either 1 or more teams?
2. Complexity:
   1. Does your team have the skills to solve the problem at hand. This may be
      rooted in how new the team is, if it is new, add a multiplicative factor.
      If it is highly technical, that is outside your immediate skillset add a
      multiplicate factor
3. External domain:
   1. Are your team developing this feature on behalf of another domain or
      product?
4. Team size:
   1. Do you either have a too large team, or a too small one. This may seem
      counterintuitive, that is that in some cases it is preferable to have a
      small team to be agile, but for other kinds of projects, getting a good
      culture and collaboration can take longer than actually developing the
      project.

I'd go as far as If you'd estimate a timeline initially for the project you
should multiply it with each of the above.

```
4 weeks * 1.5 ^ (1 + 1 external parties + 1 external domain + 1 small team)
= 4 * 1.5 ^ 4
= 4 * 1.5 * 1.5 * 1.5 * 1.5
~= 20
```

The above may seem extremely, but if you need to involve that may parties there
is a high likelihood of _anything_ splitting the time, a simple
miscommunication, etc. The above is just a naive example, but I've seen big
projects exceeding timelines by years, because of complexity of collaboration or
purely technical reasons.

In my experience you don't get a `big project` unless you've got at least one of
the above, I will dive more in depth into each, but think of it like this; a
project has multiple stages of development each of the above will add roadbumps
at each level, and to learn how to navigate these requires experience,
leadership and persistence.

### Why big projects can feel like pulling teeth out

I don't think I've ever meet a person whom likes these long drawn out projects,
fraught with extended deadlines, problems, conflicts and whatnot. In this
section I will dig into my experience of why this happens, and tied to the
previous section, on how to navigate this space, and keep a cool head.

Like many things, in acquiring knowledge about a subject, it is not a straight
`sin` curve, it is a journey filled with entropy and seemingly random events,
which are frustrating and causes problems.

Like the `Dunning-Kruger` effect it follows something like this

- Stage: 1: the project has been pitched
  - At this point you feel: Optimism, confidence and hubris; You've got this,
    you and the team are just beginning to collect requirements (hopefully), and
    everything looks good, it may seem a little overwhelming, but you feel like
    it is gonna be great, you've learned from the past projects and this is
    gonna be a breeze.
- Stage 2: The requirements just keep coming in, and nobody wants to sign off on
  what the scope is:
  - At this point you are still in good spirits, you've maybe just finished the
    first part of your PoC, and have a plan to solve the initial requirements.
    You and the team are beginning to get frustrated with the ever expanding
    requirements, and meetings. But you're still confident you can solve the
    problem within a reasonable amount of time
- Stage 3: The requirements just don't stop, meeting with stakeholders, doesn't
  go as expected, major delays from third party vendors are happening, sister
  teams have shifting priorities, it is impossible to get sign off on the
  project and your solution is half baked at best. You also lack key personel to
  finish the job in time:
  - At this point you feel burn out, exhaustion, and an assurance that you're
    not go finish in time, however at this point you communicate that you can't
    make the scope, and with the stakeholders you have a crisis meeting where
    finally, you decide on a minimal set of features required.
- Stage 4: the project is still half baked, and you continue to face problems,
  but you have confidence that the problem can be solved, you have pretty much
  full domain knowledge at this point, but the sheer amount of work required is
  daunting. Now you just want to get over with it.
- Stage: 5-: At this point either you round off the project, or you continue in
  smaller iterations of the previous stages, it depends on how long the project
  is and what the scope was decided to be.
  - At this point you will discover flaws in your solution, but no allocated
    time to fix it. You may have decided with your project owner to hack
    something together to finish the project in the earlier steps to finish the
    project in time. But now you don't have time to go and fix it. The cycle
    either continues with another project, or the same project in another
    iterations.

The above bullets are a bit extreme, but I've faced variants of these in nearly
all of the bigs projects I've seen. Some have been easier to deal with than
others, but all in all, experience is what counts. It is also why some engineers
seemingly have a higher titles than others, you may feel like your technical
skills are on par. But what counts is that you help navigate these projects on
behalf of your team, shield them, guide them and not least mentor them. (Thanks;
Anders Keller Carstensen, Martin Anker Have and Bjørn Hald Sørensen)

It is always easiest to see these projects in hindsights, but the most valuable
thing is to identify when you are about to enter one of these, so that you can
come off to the best start to them as humanly possible. You will not be perfect,
but doing the best work you can and take the conflicts required, will help
mitigate a potential disaster.

## Experiences

In the next section I will dive into how to mitigate each of the above, it will
not be a complete list, as my experience isn't complete. But it should still be
useful, it also won't be bullet point by bullet point, but rather some specific
topics which should help solve key problems.

### Requirements, requirements, requirements

Requirements are the absolute most important part of a project, it doesn't
matter how much of an experienced or skilled engineer you are, if the
requirements aren't clear, you are either toiling away for no avail, or trying
to built the tower of babel. It is not gonna happen.

Requirements seem like such a bening thing, but the difficult part of gathering
requirements are two fold;

1. You don't have complete knowledge
2. Requirements are decided by a non-technical partner

You don't know everything initially and discussing solutions with a
non-technical partner is a learned skill, that is difficult to acquire, you've
got to be confident in your teams solution, and methodology, but also being
steadfast and be an effective communicator with a non-technical partner.

My best advice here, is to exist as a unit, you may decide to be the person to
gather the requirements, gather the responsible people in a room or meeting,
poke for the requirements over several interactions with a clear agenda. Get
consensus, and clarify twice or thrice. And if you're being pressed to agree on
a solution, use `we`, say you need to discuss it with the team, and get back to
the steering committee, for the project.

An example could be:

> Business person A: We need to have National login implemented within 2 months,
> otherwise people cannot login to our platform

> You: It will potentially require a lot of work, I will need to discuss with
> the team, to tell whether it is possible or not

In the past, I would try to estimate this on my own, and give an immediate
answer. _Don't_. Whatever you give here, will set the wrong precedence. Work as
a team, and rely on them. If you are the sole technical lead, discuss the best
course of action with your team, manager. There is no silver bullet here, but
for software the best approach is to say that it is possible within a given
criteria, you may need extra hands, extra focus, or simply that it isn't
possible.

We are engineers not magicians. Don't present a concrete answer to a question
you don't have an actual answer for yet. It always ends in grief.

### The interface is everything

If you need ongoing consensus between teams or external parties you are doomed
from the start. If you need to continually interact with a second/third party to
develop your project, then you will move at a snails pace, and won't be able to
deliver on time.

The best strategy is:

- Sit together
- Have tech leads collaborate continually
- Scope the project differently

If you can't do either of the above the project simply isn't possible or sound.

The first is ideal, if you need a delivery to another team, actually
transferring that domain knowledge to them directly by peering is the best.
Usually, because of business politics and bureaucracy it isn't possible, sadly.

The second is a stop gap, and will take away resources from the team, this only
works if the techlead isn't the only person on the team with the technical
experience to drive a project. It requires a lot of work from said person, but
can make the project succeeded. It is usually done by having the two teams
senior technical leaders communicate on requirements day by day.

The third is thinking critically, can you scope the project in a way that
another team doesn't need a concrete delivery by the team you're on. This may be
deciding on an early interface or design between the teams, and then
implementing it afterwards. This usually requires follow up of either of the
previous points as well, but on a lesser degree. Requirements will change and
you will need to change the design on an ongoing basis, so plan for the
communication.

### The conflict not taken

As engineers it can be damned difficult to take a crucial conflict. This is both
a learned skill and a natural inclination. Some people are just willing to fight
tooth and nail for a solution, and others don't even want to look a stakeholder
in the eyes.

This is a make or break moment. I don't have this natural inclination for
conflict, but have learned to follow through and rely on certain techniques to
make it easier to actually remain steadfast and resolve said conflicts in a
respectful way.

If you are new, you can be owned by a seasoned business professional, be aware,
and use the `we` technique described above. Sometimes other engineers or
business professional can step over the line, if they do so, you should rely on
your leadership. This may be either a manager or director. Sometimes you may
need that distance and that is okay! especially as you learn to tackle these
things.

### Why are we doing team meetings all the time

A problem requires a certain set of skills and conditions to solve, and it is
likely that your team dosen't fit the bill perfectly, you may either be too
small, too large or don't have the technical skills to give a proper evaluation
of the project. I define a small team to be 2-3 people, and 5-10+ to be a large
team.

A too small team means that you will be impacted greatly by changes to the team,
or simply a lack of hands or capability to solve said problem. If this is you,
you need to set the correct expectations to your stakeholders that the solution
will have an undefined deadline. Let your leadership have the word here. It can
be very difficult to communicate this is in a good way.

A too of a large team: funnily enough this can be even more difficult to solve
than a too small team. It is not actually as rare as it might seem, but given a
large team, that wants to act as a unit. A consensus is often required. It can
take a long time to get consensus for even simple ideas, or features. My
solution here is to simply to communicate that we shouldn't have consensus for
everything. And rebuke the notion that you need a scrum master or agile coach or
whatever to drive your teams culture. It is fine to have team events, encouraged
even, but don't fall for the company culture events. It is bullshit in my
experience, you learn by working together, and not all at once. Work on problems
1 to 1, and solve problems that way. Large discussions can be had, but if you
decide to open said discussion be prepared with what you think will be an
acceptable answer beforehand and be ready to defend it.

### Damn, this project is drawing out

This will never become a solved problem, but you may become more equipped to
handle it. A project will have many cycles, over months, years, weeks. It can
change at a flip of a coin or be drawn out until you don't feel like you're
making any progress anymore.

This point is the simplest and hardest at the same time. This means that you
will gain the experience to face these projects in time, but will never find the
correct tools to fix them. We're sometimes prone to refine the tools for the
project we just solved, but those tools won't be sufficient to handle the next
project on the pipeline. Because if they did then it wouldn't be a big project.

With more experience you will to guide your team and yourself through this these
phases, you won't be unaffected, but you've been here before. So you now what is
going on in the trenches, but you're still in the trenches. It is a constant
balance, and the most important is to listen to your team, we each have
different capacities, and we have different breaking points. Be aware, that
especially Junior developers should be shielded from too much complexity, but
also that stress needs to be manageable for everyone.

### Deadlines, always deadlines

A deadline is such a frightening thing. It is a target, and guillotine at the
same time. But from experience that is not actually true, at least not for
reasonable organisation. A deadline is a target, yes, but not a guillotine.
After a few projects you will notice some patterns. A project can always be
extended given proper cause. It sucks having to say that something cannot be
done in time. But it has to be done.

As I always say to the teams I am working on, a project has three levers you can
change:

1. Scope
2. Resources
3. Time

If requirements are set in stone then scope cannot change, if you cannot get
extra headcount then resources cannot change. As such the only lever you've got
is time. Time is your last resort. But it is possible to extend a deadline. Have
a talk with your manager / product owner, often it isn't as bad as it may seem

### I only see walls

Tunnelvision is a real thing in these projects. When you've worked on them for a
long time, you lose your objective sense of how important the project is, how
valued it is, how well your solution fits, etc.

There isn't a concrete solution here, other than being aware, that if your
project isn't the business' primary goal, then it won't get extended funding, or
that the deadline isn't as important as it may seem.

If it is as important as the business say it is, they will throw money at the
problem to get it fixed. If not, then it isn't as important as it may seem to
you. This can be a bitter pill to swallow, as the project you've toiled away
for, may not actually bring the value to others as it initially seemed.

This is also why it is great to work iteratively, but never giving up quality,
solve a problem just as good as you can but not better. I've never seen
gold-platting giving any benefits in the past, other than having to break it all
down and starting from scratch.

### We'll do this now, but fix it later

This is a lie we tell ourselves when giving estimates, we'll just do this little
hack here, and then get back to it next quarter to fix it. If this is what you
tell your business leader, in 90% of cases, your team doesn't have the maturity
or agency you think it does.

You won't get back to fix this dirty hack, it will be a stain on your conscience
of the next years, and the business won't care. If you can't professionally
agree with a solution then don't propose it. If it sets you back 10 steps, and
moves you forward 1.5, is that really a win? no.

### Communication, I'd rather not

Communication is the most vital part of a big project. You'll have stakeholders,
and partners. There is a fine balance between oversharing, and undersharing. You
may choose to keep key learnings to yourself, because you know that it is gonna
risk the deadline. Or you may overshare causing teams to have `collaboration`
meeting each week, with no direct benefit.

_Do not_ plan regular meetings unless many teams need to succeed on a high level
task. Do share your learnings internally and ask for feedback from your manager
/ product owner. They will need the information to share more broadly, and if
you have to do it yourself, know the recipient. It is a learned skill to be an
effective communicator. One which, I've struggled with my entire career at,
sharing concise information in a language that the recipient can understand and
act on us crucial.

I will go into this in a future article for self development in this space. Just
know that all communication has to be intentional, and targeted at a specific
reader, and you're off to a good start.

### I can't do this anymore

It may not be in every project, but especially for newer teams, someone will
break. Whether it is stress, not the right expectations, giving up, etc. It is
important to listen to yourself, and the team. We don't have the same ideals,
work ethic, interests, etc. As such you may feel that the project is bearable,
but another may be on the brink. Make sure you communicate, if you're the
technical leader, make sure to set the right expectations of your teammates,
they are not you. Don't project your skills onto them. You may have a vision for
the project, but they might not know or have the skills to implement it.

Doing these projects is much more a team sport than a personal one. You will
find that as a technical leader, you will spend more and more time,
communicating and mentoring rather than implementing. As such, keep your ear to
the ground and learn to mentor instead of ripping away the keyboard from your
teammates. They are not you!

If you're a junior, if you see your tech lead being stress. Open the
communication as well. They may be bottling up all the technical problems to
themselves. They can break just as well as any other.

I've been in either situation and it is just as hard. But I've never had a bad
experience asking a person how they felt. And if they rebuke you, then the
company is probably not one that values you.

# Conclusion

In this article I've shared some of my thoughts on how big projects are being
done, and some of the experiences and learnings I've faced while doing them. It
is by no means a complete list (and if you feel something is lacking reach out
at `contact@kjuulh.io`, I am also available for general mentoring as well). But
it should hopefully give some ideas, and help newer engineers navigate the
difficult space that is project development and collaboration