In reality, businesses have to juggle tradeoffs, deadlines, and limited resources, and “it’ll get done when it gets done” doesn’t help decide whether to build feature A or feature B. Or how many people you’d need to finish the work by May.

That said, getting a decent estimate is hard, especially for a multi-quarter project. I’ve tried a number of things over my 15 years in software, with results ranging from spectacularly wrong to actually not too bad. And, based on the last few years, I think I’ve settled on something that works OK-ish.

This involves two components:

• Using the right methodology to get to the initial guess, and

• Layering in the right amount of uncertainty.

This post is meant to be a companion piece (sort of Part 2.5) for my Intro to Project Management for Developers series, though you don’t have to read the rest of the series before reading this post.

What doesn’t work

There are two approaches that I’ve seen fail repeatedly, both producing wild underestimates.

The first one is: any guess that’s based on initial gut feel, especially when not backed by experience delivering software end-to-end. I’ve often seen it from “business”-side people who don’t know what it takes to deliver software. But it’s also very common among developers who haven’t managed a project end-to-end, because they picture just coding the core functionality changes and miss the other 90% of the work needed to complete the project. This is how you get an estimate of “a couple months” on a project that ends up taking a year.

The second approach that doesn’t work, surprisingly, is breaking up the delivery into small steps, each a few hours to a few days long, estimating them individually, and then adding them up.

Well, actually, it works pretty well for projects that take a single developer a month or so to complete. But it often fails for larger projects. This happens for a couple of reasons:

1. No matter how thoroughly you design and plan the project, you will always have course corrections, inter-team frictions, missed or changed requirements, and other surprises that consume a lot of energy but are difficult to size upfront.

2. Sizing this way is a lot of work. Most people get tired after they size the first couple sprints, give up, and go with their gut feel for the rest of the project. Kind of like drawing this owl:

   

To be clear, with the second approach, you’d still likely get a better estimate than with the first. But unless you’re really thorough, don’t be surprised if your estimate is off by an order of magnitude.

So, how do you estimate a large project?

The best method I’ve found so far is to use comparable projects. This is supported by research (e.g. this and this) and has been replicated across industries—for example, by Flyvbjerg and Gardner in their book How Big Things Get Done.

The approach goes something like this: suppose you’re estimating a new feature that’s similar to one you built last year. Last year’s project took six months with four developers, or about 24 developer-months total.

Say about a quarter of that effort went into the general feature code, while the rest went into integrating with six banks, each with its own quirks and idiosyncrasies. Your new feature touches the same front-ends and back-ends but only needs four bank integrations instead of six.

A rough estimate for the new feature could then be:

• General feature work: ~6 developer-months (same as before)

• Bank-specific work: ~12 dev-months (4/6 of the previous ~18 dev-months)

• Total: ~18 dev-months

You can refine this further; for example, if you’re really confident that you can reuse some bank integration code and save one dev-month, feel free to subtract that. Though in practice, the uncertainty on this estimate will be at least ~25% (see next section), so there’s no point in stressing over any tiny adjustments.

The main benefit of this approach is that most of the time it automatically accounts for the course corrections, inter-team frictions, and other surprises that you’d have a hard time sizing when estimating bottom-up from small parts.

But what if you don’t have such a great comparable? Then you’d have to reach for less similar projects and add appropriate adjustments. Or maybe look at other teams’ comparable projects, or other projects from your experience at other companies. If all else fails, you can break up the project into coarser chunks and estimate each chunk using comparables. But be aware that once you do that, you go back to missing some of the surprises that inevitably arise when you try to fit the various parts together.

Converting the estimate to a range

So now you have a decent initial guess. But in reality, all estimates come with a range or an uncertainty level, and now we have to figure out the right uncertainty level for this project.

This is where we have to leave any pretense of science or rigorous methodology behind and go just off past experience.

And from my experience, I’d group the projects like this:

• Known territory: You have a solid comparable and you’re touching mostly the areas where your team regularly works. Your estimate is probably off by around ±25%.

• Something kinda new: You don’t have a good comparable, but you’re still mostly on familiar ground. Maybe you’re building a new kind of feature, integrating with teams or APIs you haven’t worked with, or setting up new infrastructure. In this case, the actual work will likely be up to 2x the original estimate.

• Outer wilds: You’ve never done this before and you had to piece your estimate together from POC outcomes or small individual work items. You’re likely undersizing by at least 4x. From past experience, a guess of 4-8x the original estimate is likely closer to reality.

With a more concrete example of the payroll feature above, I’d present the estimate like this in each situation:

• Known territory: “It should probably take 3-4 people a couple of quarters to do this, based on how long it took us to make {other feature}.”

• Something kinda new: “3-4 people in two or three quarters, depending on what kind of surprises we encounter and what we can reuse.”

• Outer wilds: “We’ve never done anything like this. On the surface, this doesn’t look like much, but I’ve seen projects like this really blow up. I wouldn’t be surprised the whole team ends up taking a year or two to finish this.”

…with the last estimate likely leading to a lively discussion on how to limit the scope and deliver at least some sort of useful result as soon as possible.

Alternatively, you can give estimates with an uncertainty level

Sometimes, instead of ranges, you might be answering a question like “Can we finish this by March next year?”

You can apply the same methodology and check where exactly March falls relative to the estimate’s range—though don’t forget to offset it from when you’d start the work, not from right now.

Then, your responses could be something like:

• “That’s really unlikely; other similar features have taken us longer.” Or,

• “It’s plausible, but only if everything goes perfectly and the whole team works on it. It would be safer if we reduce the scope.” Or,

• “We almost certainly can, but I’d prefer to have 4-5 people on this just to be sure we make it.” Or,

• “Yes, definitely.”

Final thoughts

So ultimately, yes, the estimation is imprecise and often wrong. But that doesn’t mean that you can’t help the decision-makers by doing your best. If you ground your estimates in reality and honestly communicate the uncertainty level, you can give people useful information without pretending to know more than you do.

And that’s really the goal: not to predict the future perfectly, but to make better decisions with the information you have.

Last week, when AI wrote two of those for me and I had to review them, I decided to try a new trick.

Well, the trick is actually old; I first saw it about seven years ago when I started working as a Tech Lead at Vena Solutions. Back then, our VP of Architecture really liked refactoring and cleanup, which often produced large PRs. To make those PRs palatable to us, he organized them into neat, very focused individual commits with one specific change in each commit, something like this:

f8ee9aa   Rename class BadlyNamedClass to BetterNamedClass
18c240f   Simplify logic in BetterNamedClass.SomeMethod()
1a4acf0   Update tests for BetterNamedClass.SomeMethod()
8b10bc0   Clean up unused imports
          ...(etc)...

Some commits had a lot of lines changed, but it was the same change on each line (e.g. rename SomeThing to OtherThing). Other commits had a specific narrow focus, like a tweak to a complex algorithm. Each individual commit didn’t have to be compilable or pass tests by itself; it just had to tell the next chapter in the overall story of what the PR was doing.

You can get AI to reslice an existing PR

That’s the new trick I tried: getting the AI to do that to a pre-existing PR.

I would have never done this by hand myself. But AI has infinite patience (well, as long as the tokens last), and it had no problem tearing apart all 4k lines of code and carefully piecing them together into 20-ish commits. It wasn’t perfect, but it was good enough that reviewing was a lot easier.

So then I turned it into a skill (below); after using it on several more PRs, I think from now on I’ll be doing it every time just so that I can read the PRs myself.

---
name: restack-pr-commits
description: 'Reorganizes the PR as a series of easily reviewable commits.'
argument-hint: 'for pull request <PR URL>'
---

## Overview

The goal of this skill is to turn a large, hard-to-review 
PR into a series of commits that are each individually easy 
to review. The commits, once fully stacked, must match the 
original PR character-for-character. 

Each commit is "easily reviewable" if it introduces a 
relatively straightforward change. It can be a commit 
affecting a large number of lines that does something simple 
(e.g. rename a variable or clean up unused imports), 
or a commit that adds a specific tricky concept in one 
file, or something in between. Generally, new tests 
should be added separately from the newly added code, 
but a change to functionality can have a single commit 
spanning both the main code and the tests.

Each individual commit doesn't need to pass the build or 
the tests; the primary criterion is reviewability.

## Steps

You should have been provided a PR to work on. If you 
haven't, stop and ask for it.

Make sure that you're clear on which branch the commits 
are based on. We'll call this branch the "base branch".
This would usually be the repo's main branch (called 
`master`, `develop`, `dev`, or `main` - this depends on 
the repo) but it may not always be the case. If it's 
unclear what the base branch is, stop and ask.

1. Check out the PR's branch locally. Make sure you're
   on the latest version of it. Once done, your 
   current branch should be the PR's branch. We'll
   call it `{current-branch}` from now on. If during 
   this step you cannot use `git` or `gh`, stop and let 
   the user know, as you won't be able to proceed further.
2. Make a backup of the current branch.
    - Do: 
        - `git checkout -b {current-branch}-backup`
        - `git checkout {current-branch}`
        So that you end up with the backup of the current 
        branch, but you'll still be working on the current 
        branch. DO NOT drop the backup branch; we will 
        leave it forever as a backup, so don't clean it up.
3. Come up with the commit breakdown.
    - Review the contents of the PR, the PR description, 
      and any relevant documentation to fully understand 
      the changes.
    - Come up with several distinct ideas for how to 
      split up the PR into easy-to-review commits.
    - Come up with the criteria to figure out which of 
      the choices is best.
    - Review each of the proposals for the split. Evaluate 
      them against your criteria, critique them, and rank 
      them against each other.
    - Based on your critiques, update each of the options 
      for the split.
    - Critique and rank these options again and come up 
      with the final ranking.
    - Come up with the final recommendation for how to 
      split the PR into commits. The final recommendation 
      may be any combination of the proposed 
      recommendations based on your analysis.
4. Figure out a strategy you'll use for verifying that 
   the final result matches the current PR 
   character-for-character. Make appropriate preparations 
   as necessary.
5. Rebuild the PR as a series of commits on top of the 
   base branch as per your proposal. After each commit, 
   verify that the commit has added exactly what you 
   think it should have added and that there's nothing 
   missing and no copying errors. If there are any issues, 
   amend the commit as needed. Each individual commit
   doesn't need to pass the build or the tests, so don't 
   rerun those between the commits. 
6. Verification: when done, verify that the final 
   result matches the current PR 
   character-for-character. If not, fix any commits 
   as needed. 
7. Rerun the build and the tests as needed. 
8. Force-push the changes to the origin so that the 
   PR now looks like a series of new commits.

Do not clean up the backup branch; we may need it 
later if there were any issues.

But what about stacked PRs?

If you aren’t familiar with stacked PRs, this GitHub page gives a decent overview.

They’re certainly an option, but I find that before starting the implementation, it’s not always clear which PR should be the base PR, and sometimes after a lot of fixes and follow-up tweaks, the stack needs to get reorganized a lot, which is annoying to do when you’re dealing with PRs instead of commits.

And it’s a lot more practical to reslice a 4k LOC PR into 20 commits rather than 20 stacked PRs.

It certainly will if you just blindly merge the PRs and send them off to prod. But if you take a thoughtful, layered approach to managing quality, I find that it’s possible to not just keep the number of bugs stable but actually reduce it—while still increasing the output by 2-2x.

Many of these defensive layers are pretty much the same as before Claude/Copilot/Codex/etc. (though they’re made easier now by AI), while others are new. Here’s a defensive setup that I’ve seen successfully used in practice, both on my team and elsewhere.

Layer 1: Getting the requirements right

One of the biggest surprises after I started using spec-driven development was the drop in bugs in the freshly written code. Before spec-driven development, when building, e.g., a new feature, the teams I was on often spent up to a third of the total effort on the post-development “polishing,” i.e., discovering and fixing various bugs. Many of these bugs occurred either because we didn’t foresee certain interactions and edge cases, or because the developer was tired that day and didn’t put in enough thought, or because the designer or PM didn’t think through certain scenarios. Some of these bugs were missed and ended up in production.

After I started using spec-driven development, the number of these bugs in my code sharply dropped, and I’ve seen the same drop for some (but not all) of my teammates. As far as I can tell, the main cause of this drop is one specific step in the process: having the AI review the requirements or the tech design and find any gaps, edge cases, unexpected interactions with the existing code, or other similar problems.

The AI doesn’t get tired and, when prompted right, is a lot less likely to give up hunting for potential issues. If anything, it can sometimes be overzealous, and I have to carefully review its proposed edits to the requirements to make sure that it doesn’t invent any issues that aren’t there.

Layer 2: Unit tests at >95% coverage

Coding agents now make test-driven development (TDD) trivial to the point where there’s no reason not to do it. However, it needs to be done right: you don’t want the agent to blindly write passing tests for any bugs it just added to the code. So the best planning and implementation skills I’ve seen usually follow this pattern:

• Instruct the agent to think through the test scenarios and test cases based on the requirements,

• Write the test cases,

• Write the implementation,

• Test the implementation against the test cases and fix any issues that come up,

• Maybe backfill any remaining coverage gaps—but again, keeping the requirements in mind.

Also, with the agents writing the tests, there’s no excuse not to shoot for near-universal coverage or to wait on backfilling any missing unit tests.

Layer 3: Manual testing

There’s still no substitute for a human (you, QA, PM, or someone else) actually trying out the feature, going through all the edge cases, and seeing whether everything works as expected or whether you need to make changes.

These manual tests can take a while, especially if the test scenarios take some effort to set up. This is one of the steps that so far has seen only modest gains in productivity, and it’s the main reason that my output has increased only 2-3x instead of something like 10x. Though now that I think about it, there may be a few opportunities for automation here that I’ve missed.

Layer 4: Extensive automated end-to-end tests

End-to-end (E2E) tests are arguably the most important tests in the codebase because they verify that new changes haven’t broken any existing functionality as experienced by the end user. Ideally, they’d run on the PRs, in the test/stage environments, and in production after every deployment. Ideally, they’d also be maintained by the same developers who write regular code, but I understand that some organizations aren’t really set up for that.

AI does make it easier to write E2E tests, but to do that effectively, it needs access to the tools or MCP servers that let it debug test failures—e.g., a browser tool or MCP access to the logs. However, it’s important to keep in mind that E2E tests aren’t a substitute for manual testing because they’re just a rough, incomplete check that nothing important broke.

Layer 5: Code quality passes by AI

I find that coding agents aren’t great at following complex instructions in AGENTS.md or CLAUDE.md. But they do pretty well if you add a separate pass to find and fix specific issues. These can be:

• Security issues,

• Finding overcomplicated or duplicated code,

• Compliance with naming, file organization, or formatting rules,

• A general code review pass to find any issues with the logic,

• Overly long comments written in AI-ese instead of regular English,

• Any other specific things that you’d like to find and fix.

If added to the planning or implementation skills, these can be pretty much “free” additions, adding maybe 5-15 min to the implementation time with no additional attention required.

They can be also added to the PR reviews if you prefer to take a look at the comments before applying any fixes.

Layer 6: PR reviews by humans and AI

I think I’m becoming convinced that for minor tweaks and simple bug fixes, human reviews can become optional. Provided that other defensive layers are still in place.

But for complex changes, I find that it’s still necessary to review AI-written code. I still regularly find big-picture mistakes, missed adverse interactions with other features, overcomplicated or suboptimal implementations, and other problems. Not to mention weird word choices like “mint” instead of “generate” or “stamp” instead of “set.”

AI code reviews have also been a really great addition. On my current team, we run both Claude and Cursor reviews on the PRs, and surprisingly, each of them finds different problems. You can also add other custom reviews from various angles, like security, efficiency, interactions with other repos, and so on, though be aware that AI can be overly nitpicky in its reviews, so it’s important to also have a pass where another agent prunes the proposed AI-generated PR comments that aren’t actually meaningful.

Layer 7: Monitoring and alerting

Once the code is in production, at a minimum, it’s good to have someone periodically scroll through the logs or watch any user recordings in something like Fullstory, or review various dashboards that track error rates, latencies, and other issues.

Even better would be an error tracking service like Sentry or GCP’s Error Reporting that detects and deduplicates errors.

The best approach, however, would be to then have Claude/Cursor/whatever auto-diagnose these errors, figure out the root cause, and make PRs with the proposed fix.

Conclusions

I’m sure I’ve missed other important components of maintaining high quality, but the main idea is that with the right set of defensive layers, the increased output doesn’t have to come at the cost of reliability. If anything, coding agents now make it cheaper to add more and deeper checks than before: more tests, more review passes, faster diagnosis of production issues.

So if you’re sufficiently focused on quality, I think it’s entirely possible to double the delivery speed while keeping the bugs under control. Or maybe even reducing them.

You’re starting to get annoyed, but you aren’t sure what to do. Should you tell them off and cause a confrontation? Send them a link to noslopgrenade.com or dontpastetheai.com and come off as passive-aggressive? Politely try to hint that maybe you were hoping for a different answer?

I’ve run into this situation a few times and I think I have an approach that works reasonably well.

Warning: this may not work for everyone

Admittedly, this approach works best if you’re the person’s manager. And you should probably avoid it if the copy-paster is a thin-skinned person who holds a lot of power over you.

Between those two extremes, you’re more likely to succeed if you’re in a more senior position, or have accrued credibility with the person, or if your coworker has signaled that they’re very open to feedback.

First, identify the actual problem

Ask yourself: what specifically about the incident caused you inconvenience?

Was it that you had to read through a wall of text when a single sentence would have been enough, wasting your time and attention? Was the document confusing to read and follow? Did you have to wade through 10 comments on your code that obviously missed the context? Is it that you were hoping for their own opinion but now you aren’t even sure they’ve read the question?

Sometimes copy/pasted AI output isn’t actually a problem. For example, a couple of weeks ago, a co-worker of mine sent me a comment on my code that said “Claude Code says: {quotes Claude Code}. Do you think it’s worth checking out?” On the surface, that should have been annoying, but this coworker (1) is often right, and (2) is usually thorough in his work and reviews. So this comment was likely filtered from many others left by AI and probably contained a valid concern.

But suppose there’s an actual problem, and you’ve identified what it is.

Next, bring this up with the coworker

I usually open the conversation with something like:

Hey {name}, are you open to some unsolicited feedback on {thing in question}?”

Or some variation of that. This does a couple of things:

• The answer will probably be “yes”, and hopefully as a result they’d be at least partly open to the feedback.

• You’re setting up an expectation that there’s some negative feedback coming so it doesn’t just drop on them unexpectedly.

Next, follow the rules of constructive feedback:

• Be direct; don’t dance around the problem.

• Concentrate on the behavior and its negative effect on you rather than concentrating on the other person.

• If possible, prefer to use “I” statements like “I was confused…”, “I had to spend more time…” rather than accusatory “you” statements like “you made me waste time”. The feedback might be negative enough as it is, and phrasing it as “you’re bad” is an unnecessary extra slap in the face. Of course, sometimes it’s unavoidable.

• Keep in mind that it’s not the use of AI in general that’s the problem, it’s this specific use that’s the problem.

So your next message could be something like:

When you give me unfiltered AI output as PR comments, I have to spend time figuring out whether each of the comments even makes sense. It would be a lot more helpful if you pre-filtered the AI findings yourself and left only those that you think actually apply.

Or,

I found that the document was confusingly organized: {add specific problems}, and I had a hard time following it. I think it would have been a lot clearer if it had… {specific suggestions}. In general, I find that AI is pretty bad at organizing documents, and I have to edit them myself afterwards.

The consequences

I’ve had various responses from coworkers to this. One was very open to feedback, said thank you, and moved on. One was more defensive, but the feedback was eventually internalized and the issue did not reoccur.

Whatever happens, don’t get into an argument—it will only entrench the other person further in their views. You’ve left the message; you can move on now.

Now that everyone’s clear what exactly needs to be done, it’s time to start on more thorough design and planning.

I’ve split this post into two parts:

• Technical Design: addresses what the change should look like, and

• Planning: preparing to deliver the changes smoothly.

But first…

Why should you do technical design and planning?

Again, some of you may be tempted to jump straight into building and figure things out as you go. That will work for a small deliverable that you’re doing by yourself. However, if you skip the design and planning for larger projects, you risk running into problems like this:

• Halfway through, it turns out you need another team to do something, and their backlog is full until the end of the quarter.

• Mid-sprint you discover that one of the developers disagrees with you on the design. Now both of you need to stop your work and debate it. The rest of the team has to stop too because their work depends on the outcome of the debate.

• The code that worked for the first two parts of the project doesn’t work for the third, and now you need to rewrite most of it.

The job of technical design and planning is to discover issues like these early when they’re cheapest to address. It won’t prevent all surprises; you could still get last-minute feedback that the users are confused by the interface and end up having to rewrite it. But at least you can minimize the delays and rework from the foreseeable problems.

Technical design

With that in mind, when I go into technical design, my goals are:

1. Uncover as many technical surprises and as much unexpected work as possible—plus any missed requirements.

2. Uncover any hidden dependencies, especially dependencies on other teams or outside vendors.

3. Make sure that the team is clear on what we’re doing and agrees with the approach.

4. Get a better sense for the size of the work, how many people are needed, and how it can be broken up and parallelized.

We’re going to do this by diving deep into the design and, again, having some meetings.

Discovering surprises and dependencies

From everything I’ve tried so far, there isn’t really an easy way to do this except to go and design the system in detail.

To keep track of that, I usually start a technical design document. If your organization already has a technical design template, try to follow it; if not, feel free to start another document similar to the requirements one. I find the exact format doesn’t really matter, as long as the document is reasonably readable and follows basic principles of technical writing like putting the summary and most important information first, then gradually expanding into details.

In this document, start sketching out the system design in a bit more excruciating detail than you’d first think is necessary. Like, not the individual functions and classes, but maybe a level above that.

For example, in my recent project that delivered a new user-facing feature across several front-ends and back-end services, the technical design involved writing out:

• The full list of the front-ends and the full list of the back-end services involved, how each one would change, which new REST API endpoints would need to be created, request/response sketches, and what authentication would be used in each case.

• A detailed sketch of how each of a couple dozen or so user flows would be handled by the system, including how it would move through the front-ends, back-ends, DBs, auth used, vendor interactions, and so on. Since the system involved moving around and submitting complex data structures, the design also included listing where each data field would come from and how we made sure that it was correct.

• Any data model changes and data migrations, the exact list of the background jobs that would need to be present, and any notable infrastructure changes beyond the standard setup.

Plus a few other details.

Other things worth including, depending on your situation:

• Expected scale, latency, and throughput.

• Security, privacy, authentication, and permissions.

• Any dependencies on other teams or vendors/partners. You’ll want to start conversations with them early.

• Whether any organizational reviews need to be passed, such as security, legal, compliance, or architecture reviews.

What’s the right level of detail to go into when writing everything out? Well, consider the earlier goals: we’re trying to uncover major surprises like unexpected work, dependencies, or difficult technical questions. So focus on the details where getting things wrong would cause substantial rework, delay, or a dependency surprise. Don’t overdesign details that are cheap to change later.

Tracking open questions and risks

Just as with the requirements, when writing a technical design, it’s good to keep an eye on spots where you’re not totally clear on the details. This could be either questions about requirements, or questions about how to implement something, or maybe questions you need to ask other teams, or architects, or vendors.

In these cases, write these down so you don’t forget them. And make sure to follow up on them.

Also, remember the Risks section from Part 1? Now is a good time to sit down and ask yourself: what are some of the ways the project can go wrong? These could result in more open questions or follow-up meetings.

Getting the team on board

As mentioned before, one of the goals of the design process is to make sure that all team members who’d be working on the project understand how it would be implemented and agree with the approach.

The best way to do that, I find, is to have a meeting with all project participants and walk them through what you have in mind so far. Then ask for questions or opinions and be genuinely open to feedback and criticism. Take those away, think them through, and update the document as needed.

In cases of major disagreement or concerns, I found it useful to have follow-up discussions with the person who brought them up, maybe even working with them over a few days or weeks to resolve the issues or update the designs.

Simplifying the design

As you’re working through the design, it’s worth periodically asking questions like:

• Is there a simpler way to do this? Or, do I even need to do this?

• Now that I know the cost of this requirement, should we still do it?

The latter question has often led to conversations with product managers or other decision makers who, after realizing the cost, ultimately said “you know what, maybe we don’t need this.”

Planning

I usually start transitioning to planning once the design starts stabilizing, though depending on the time constraints you may have to start planning out some pieces sooner.

Each organization has its own way to track large features/epics/projects together with their subtasks, and its own processes for refinement or planning. You’re more familiar with those than I am, so I won’t give you advice on how to use them. Instead, I’ll just give some general pointers for planning larger projects.

Unlike design, I found that it’s OK not to go too deep into planning upfront and instead take it sprint-by-sprint. That said, there are still a few questions that you’d need to answer before planning the first batch of work:

1. What is the dependency between the different deliverables? Which ones are blocking—i.e. must be delivered first, before any work that depends on them?

2. Now that you know the design, what’s the total effort? Do you think you’d meet your deadlines? Should you go to your manager or other decision makers and renegotiate the scope, the timeline, or the resources (i.e. the number of people working on the project)?

3. If you have more than one person working on this project, can you parallelize things in a way that keeps everyone busy without any gaps or downtime?

4. There may still be surprises lurking; what’s the best way to uncover them as early as possible?

For this purpose, I find it’s helpful to at least do a rough sizing of the various components of the project with the team, add some padding for the inevitable problems and unknowns, and sketch out what pieces should be delivered in the first few sprints.

Integrating and deploying the work

One of the questions that also needs to be addressed upfront is what your overall approach to integration and testing will be. By that I don’t just mean CI and automated tests (I assume that’s a given these days), but for the entire project as a whole.

I’ve seen projects delivered two ways:

• Approach 1: Everyone writes various pieces separately, possibly coding/testing against pre-agreed interfaces. Then the entire system is put together, tested in the test environment, and then deployed to production.

• Approach 2: First, the thinnest possible sliver of end-to-end functionality is deployed to the test and production environments. Then everyone builds out the various pieces around it, doing integration testing in the full system as they go.

From personal experience, Approach 2 is far superior. This is for a couple of reasons:

• Deploying to prod as soon as you can allows you to do end-to-end testing earlier, show the system to the prospective users earlier, and adjust course sooner. Also, some bugs show up only in the end-to-end tests; would you rather find them early in the project, or a week before launch?

• Production deployment is often more complex than you’d first expect, especially in larger organizations and for large projects. You may have to wait for approvals, satisfy unforeseen requirements, or (in at least one case I’ve experienced) switch to an entirely new deployment method due to a recent directive by a CTO that you weren’t aware of until you tried to deploy. Ideally, you’d deal with these surprises early.

For this reason, in the first couple of sprints on large projects, I prefer to focus not just on building out the initial critical pieces of code, but also on deploying shells of all components to production and making them demonstrate end-to-end connectivity.

Pieces that often get missed

When planning the work, even senior engineers sometimes forget that “Done” is not just when the code is written or the PR is merged, but when:

• Someone confirms in the test environment that the change does what it’s supposed to do,

• If it doesn’t, any follow-up adjustments and fixes are made, and

• The change is deployed to production, where it’s also proven to work as expected.

Often these steps take nearly as much time and effort as the original code, so make sure to include them when planning both the individual tasks and larger components.

How long should design and planning take?

On shorter, simpler projects I’ve had design and planning take a day or two, plus one or two meetings with the team. For larger multi-quarter projects, I’ve had it take up to a couple of months with multiple follow-up meetings with various individuals.

Moving on

With the tech designs firmed up and the initial work planned out, now it’s time for the fun part: building.

• Next up, coming soon: Intro to project management for developers: Part 3 - Staying on top of delivery

In this series of posts, I’ll introduce a few informal project management techniques that should help you do better. These are based on my experience in the last 15-ish years of leading various software projects in four different companies on teams ranging from internal software to platform engineering to SaaS product teams.

The techniques will specifically address:

• Making sure that everyone is on the same page—including you.

• Discovering dependencies, risks, and surprises early.

• Producing a reasonable effort estimate.

• Staying on top of delivery and having it go smoothly.

• Having a smooth rollout.

All of this won’t fit into one post, so I’ll break it up into three parts:

• Part 1 (this part): Initial notes and kicking off a project

• Part 2: Technical design and planning

• Part 2.5: Estimating software projects

• Part 3: Staying on top of delivery (coming soon)

But first…

What’s a “project”?

For the purpose of this post, it’s a chunk of work that:

1. Has a clear “start” and “done”, like a new feature, an infrastructure upgrade, a refactor, or a minimum viable product for a new system; and

2. Has a delivery time between a single engineer working for a month on the low end and a team of 5-7 working for a couple quarters on the high end.

Different organizations call this type of work by different names: epic, initiative, feature, or something else. For the purpose of this post, I’ll call it a “project”.

The overall principles from these posts would still be useful for work outside of this definition, though the specific techniques would be overkill for a week-long task and insufficient for a multi-year cross-department program.

Also, every company, team, and project is different, and what you’ll work on is not what I’ve worked on. So treat these posts less like rigid step-by-step instructions and more like a toolkit or a starting point for your own process.

With that in mind, let’s begin.

You just got a new project. Now what?

As a software engineer, your first instinct on a new project may be to start designing a solution or thinking through the code you’d write.

Suppress that instinct.

Instead, the first thing you’ll need to do is make sure that:

1. It’s clear why the project is being done and what success looks like,

2. There are clear expectations for what exactly will be built, and

3. The right people agree on the direction.

If you don’t do this, you might confidently deliver something that no one needs, and you’ll discover that only late into the project.

The rest of the post will focus on how to get those three things done. To do that, we’ll need to start one document and have a few meetings.

Step 1: Start a project outline document

To begin, we’ll need to document what we know about the project so far, and what kind of questions need to be answered. A great way to do that is to start a project outline document.

It can be a Word/Google doc, a page on the internal wiki (like Confluence), a PowerPoint deck, or any other document; the main criteria are that (1) it should be whatever your company uses for similar documents, and (2) bonus points if it can be easily shared and collaborated on.

In it, add several blank sections:

• Overview

• What’s out of scope

• Requirements (feel free to rename it to “Details” if you or your org are uncomfortable with the word “Requirements”)

• Open Questions

• Risks

Don’t worry too much if you cannot complete—or even start—some of these sections until you’ve spoken to some stakeholders1. But do try to fill out as much as you can eventually.

Here’s what goes into each section.

“Overview” section: add a couple paragraphs with the project motivation for those who’re new to it, what goals it’s trying to accomplish, and what success looks like. It can also include a brief summary of what exactly would be built.

“Out of scope” section: clearly state what the project will NOT deliver, or what may be delivered later but not this time around.

“Requirements” section: this is usually the biggest section. In it, outline what you think the project is supposed to do, how it’s supposed to interact with any existing features/components/systems. For user-facing features this could also include outlines of user flows; for internal migrations or upgrades this could include the end-state vision and list of systems to upgrade. And so on.

You don’t need to include the fine details like the exact UI fields and validation error messages, but it should be clear enough that by the time you get to working on a detailed technical design, there are minimal surprises.

This is also a good time to look around and do a quick gut-check of whether the project is even possible by briefly taking a look at the code (probably with AI’s help these days), or reviewing the documentation for any external systems you’d be interfacing with (add a link to these in the doc). It’s also a good time to note which parts would be responsibilities of another team. Feel free to start sketching out the implementation too (see Part 2 for details), but don’t go too deep into it—we’re not at that stage yet, and a lot can still change.

“Open questions” section: As you write out the requirements, you will inevitably run into spots where you’re not sure what exactly the expectations are. In those cases, stop and leave a bullet point in the “Open questions” section noting what should be clarified and who it should be clarified with. Alternatively, you can leave comments straight in the document, or you can add questions inline in a different color (like this?).

“Risks” section: you can fill this out later during design/planning, but as you go it’s good to ask yourself what can go wrong. Usually it’s questions or concerns like “we probably can’t deliver X by the end of the quarter with only two people”, or “what if their API doesn’t support this?”, or “is team Y willing to deliver this by the end of the next month?”. Some of these can be answered by looking up the relevant info right away (e.g. the API question), but for others, you’ll need to follow up with the right people—more on that later.

Step 2: Talk to the stakeholders

Next, it’s time to book some meetings with people who can give you the necessary information. This can include the decision-makers like your managers or clients, subject matter experts in the relevant domains, or even developers on other teams who can explain how their systems work.

The main goals of these meetings will be to:

1. Clarify what exactly needs to be delivered,

2. If you’ve written down anything in your project doc, confirm that it makes sense and is on the right track,

3. Answer any open questions you have, and

4. With the main decision-makers: establish the priorities, especially if some parts of the project cannot be delivered on time or will require more resources.

I usually first meet with the people whose opinion matters the most, like my boss or the product manager or the client, and then work my way through everyone else.

You can also try to do this via chat or emails, but I find that at least at the start, it’s better to have a meeting because asynchronous communication doesn’t work as well until after the common expectations have been established.

Update the document as you learn new information and as the open questions and risks get addressed. Add the new risks and open questions as needed.

Descoping and reprioritizing the work

Sometimes it becomes clear that some things are bigger than originally thought or that they’re not realistically deliverable. These can become candidates for “descoping”: reducing the expectations or eliminating them entirely.

If you’re the full owner of the project with the power to make decisions on what gets done, then go ahead and cut or re-prioritize as needed—keeping in mind the project’s ultimate success criteria. Though depending on where things stand, you may need to have a chat with some stakeholders first to get their opinions or at least give a heads-up.

If someone else is the ultimate owner, I recommend against just saying “we cannot do this” or “we won’t do this”. Instead, I suggest stating the constraint and offering choices: “I don’t think we can realistically deliver both X and Y, which do you think should be a priority?”, or “there’s a high risk we cannot deliver X on time with two developers; do you think a third can be added? Or are we OK with X going into the next month/quarter/milestone/etc?” Then let them make the decision.

Anything that ends up being cut, moved, or deprioritized should be added to the “Out of scope” section in your project document together with any appropriate notes.

Handling disagreements between major decision-makers

Sometimes several important decision-makers disagree on key details or priorities.

In this case, the approach that worked best for me is to book a meeting titled something like “Agree on (topic of disagreement)” and invite all decision-makers—or, at least, the disagreeing ones—and let them discuss the issue and come to a conclusion. In my experience, they will almost always come to some sort of understanding. If they still disagree, the next steps are highly dependent on your circumstances; it can be anything from escalating the question to upper management to giving up on the project altogether.

Last step: Final alignment

Once most of the project details have been nailed down, it’s often useful to have one additional meeting with all the decision-makers and project participants where you’d go over the final outline of what will be done and what will not be done.

Depending on how your organization works, this can be either a dedicated meeting or it can be part of another meeting like bi-weekly refinement. Or you might even be able to do it asynchronously via email or chat. Though I do find it helpful to have everyone in the same room for this step.

Keeping the project expectations up-to-date

You now have a set of initial expectations for the project that everyone understands and agrees on. However, don’t be surprised if things change, especially if this is a larger project with a longer delivery time. When that happens, sometimes all key decision-makers will already be aware of it and in alignment, but sometimes they won’t. In that case, it’ll be your job to have any follow-up meetings or discussions to make sure that everyone is on the same page about the new project direction.

Moving on

Now that we’re clear on what’s to be done, we can move on to the next step: detailed design and planning.

• Next: Intro to project management for developers - Part 2: Technical design and planning

Or this (link to post):

I understand why people have vent at Claude/Cursor/whatever when it screws up. It talks like a person, so when it messes up, it’s tempting to tell it off, maybe even add a swear word or two.

However, I think not a helpful approach. Instead, when that happens, pause for a sec, and consider this:

• The agent is going to forget the conversation once the session ends. Well, unless it has memory of some sort, in which case it might remember the wrong lesson.

• What do you want the outcome of the conversation to be? Will yelling at the agent solve your problem? Will it prevent the mistake from happening again?

• The agent is not a human. It’s a program that just demonstrated a bug. And you’re a software engineer. Why don’t you do some debugging?

So rather than typing “what the fuck did you just do??”, try some version of this prompt:

It looks like after the last prompt, {describe the issue}. Investigate why this happened and try to find the root cause. Suggest approaches to avoid it in the future.

Or just step back and think through how the issue could have happened—based on what you know about LLMs so far. Could it have been context rot from too many instructions? Or was important context not added somewhere? Should you have used a smarter model or set the reasoning level higher? Or maybe you need to update the rules in AGENTS.md/CLAUDE.md?

In the last year, I’ve now run twice into a situation where I was seriously tempted to type “WTF???” as my next prompt. However, after doing some investigating, the first issue led me to change the way I prompt and write skills, and the second issue turned out to be a bug.

Finally, keep in mind that LLMs are non-deterministic systems, and there’s an inherent risk that sometimes they’ll do something random and stupid. So it’s a good idea to put some boundaries in place to prevent them from causing real damage, such as blocking force-push on the repo’s main branch, or blocking write access to the prod infrastructure.

And if you want the agent to code a complex feature end-to-end, you may want to add another step before planning: a requirements and design document, also called a “spec” file.

For me, once I’ve dialed in the technique, these spec files have been responsible for the single biggest productivity bump I’ve ever had. Looking back at the features and projects I’ve delivered since I started using them, when I compare the old estimates with the actual delivery times, it’s safe to say that my productivity rose around 2x while improving the quality—as observed by the number of bugs found by our manual QA and the number of post-deployment fixes (more on that here).

But how exactly should you write these spec files to get the most out of them? I’m sure many people have arrived at their own approaches, many of them better than mine, but here’s how I do it.

Note that I use Cursor (at work) and VS Code+Copilot (at home), but in theory this process is very harness-agnostic and should work with most agents.

Step 0: the template

The first thing I do is copy+paste the template below into the working folder and fill out the obvious parts: the title, the overview sentence, and a rough sketch of the initial requirements. (By the way, if the template looks suspiciously like a miniature version of my method for planning larger projects, that’s not a coincidence).

Note a few blocks of text that are intended strictly for the agents that will be updating this file:

• Document rules: deals with a variety of annoying quirks of various models.

• High-level design: contains a sentence telling the agent to keep the designs high-level.

• Open Questions section: contains rules for updating the open questions.

The rest are the instructions and examples for you, the human driving the AI.

# Title - short description of the project

A statement in a sentence or two describing what you'd like to do.
Example: we'd like to add the ability to send email notifications in our system.

## Document rules

Do not delete this instruction paragraph. Follow it carefully when editing 
this document. This file should contain final designs only; do not track 
changes or discuss pros/cons of alternatives, or discuss previous 
implementation plans unless explicitly prompted. Avoid leaving large 
paragraph of text, they're hard to read; find another way to organize 
text if necessary. Do not use headings past h3 (`###`), find another way 
to organize text if needed. Keep changes brief and to the point - avoid 
adding any text that doesn't need to be added. Avoid large blocks of code, 
though feel free to add focused blocks of code if it really helps 
communicate the design. 

All target designs/future designs should use either future tense ("X will 
use...") or describe the change to be made ("Update X to..."); present 
tense ("X runs only on...") should be used only when describing the 
currently implemented designs. Using present sense for future target 
states may confuse the reader.

## Requirements

Outline the requirements. Feel free to break them up into subsections 
if needed, or can just start in point form and expand later.

Example: 

We'll want to send emails on:
- Account creation
- Account deletion
- Manager-initiated account activation
- Manager-initiated account deactivation

New accounts are created through self-serve signup only, and account-creation 
emails are sent for those self-serve signups.

For account deletion, send the email for both self-serve deletions and 
manager-initiated deletions.

... And so on.

## High-level design

To be filled out later.

This section should contain high-level designs only. Avoid detailed designs, 
that's for the planning/implementation agent later.

## Open questions

- **Example open question**: Example open question details.

Do not remove this section or this instruction paragraph. If 
you're removing the last open question, add a correctly formatted 
example placeholder question so that future agent runs can add 
more open questions if they need to. The placeholder should 
be: `**Example open question**: Example open question details.`

By the way, feel free to add or change any sections as needed. For example, when refactoring, I like to add another section called “Current design” and rename “High-level design” to “Target design”.

Step 1: Requirements

Now is a good time to pause and think through the requirements a bit more, filling them out as much as you can.

Here are some of the things I do next:

• Talk to my product manager, the designer, or any other relevant people to refine the edge cases and clarify specific behaviors.

• Get mock-ups from the designer, take the screenshots, give those screenshots to my coding agent, and ask the agent to (1) describe what it sees, and (2) add those details to the requirements section. (I usually ask it to describe the changes in terms of any existing components in the codebase as much as possible.)

• Grab any third-party docs I need and either reference them in the spec file, or get the agent to summarize them.

• Ask the coding agent to review the requirements for any inconsistencies or gaps.

For the last point, there’s a specific prompt that I like to use:

We’re working on `my-spec-file.md`. Do the following:

* Review the file and any relevant documents and code. Think through how you’d implement it. Are there any gaps in the requirements? Any conflicts? Other concerns? Any unexpected interactions with existing functionality? Emit a list of issues. We’re looking only at the issues with the requirements at the moment, not technical design.

* Review each issue from the previous step. For every one that is actually a non-issue, ignore it. For any issue that has a single obvious best answer or any that can be answered with “just do what the code does now”, update `my-spec-file.md` to reflect that. For all others, add them to the “Open questions” section.

In a system of any complexity, this will almost always find something to add to the document. And a smarter model (like say, Anthropic’s Sonnet/Opus, or OpenAI’s Terra/Sol) may also raise some open questions that I haven’t thought of.

Though sometimes they raise dumb questions too, or make wrong edits.

Step 2: Carefully review the updated requirements

If you usually skip reviewing AI’s output, I won’t judge you much (I’ve done that at times), but this is really not the time to do it. At the requirements stage, a wrong assumption by AI will take 2 min to detect and fix. If it’s missed, it could cost you 30 min of rework at best or become a production incident at worst.

Think about it like this: you’re ultimately responsible for these requirements. The AI can help you, but it just cannot know as much about the requirements as you or the humans around you.

I usually do a couple of cycles of prompt for gaps/open questions → review → apply fixes/answer questions → prompt for gaps again.

Step 3: High-level design

Once the requirements have stabilized, it’s time to move on to the design.

On simpler features you can just directly prompt for one, but more often than not I do something like this:

• First, I put down a few technical decisions that I know upfront: e.g. use a specific library, or a specific third-party service, or that service X should own objects Y. This is also a good time to add sizing constraints: should scale to at least 2 million of {whatever object} across N users, should have p95 latency <500ms, and so on.

• Second, I usually have a few open questions of my own, like how to design a certain part, or whether a certain thing is even possible.

• Then, I systematically work through various parts using the /tech-design skill below:

---
name: tech-design
description: 'Come up with a technical design for a problem. Use when asked a challenging technical question or to design a solution for one specific problem. Researches best practices, generates options, evaluates tradeoffs, and recommends the best approach. Best for one focused problem — invoke separately for each independent decision.'
---

# Come up with a technical design for a problem

Considers and selects a technical design for one problem. 
Best for one specific problem, not when there are multiple 
independent decisions to be made. If there are multiple 
independent decisions, it's best to use this skill for each decision.

## When to use

- Use this skill for answering a challenging technical question 
  or coming up with a technical design for one specific problem.
- Avoid using this to solve multiple problems at once. If there 
  are multiple independent problems/questions, it's best to invoke 
  this skill for each question/problem separately.

## Instructions

- Read the relevant documentation.
- If there are AGENTS.md, CLAUDE.md, cursor rules, or any other rules 
  files, review those.
- If there is code to review, review the relevant code.
- Research online to see what other people have done, what the best 
  practices are, any insights from blogs, research papers, or people 
  who've done similar things before.
- Think through the problem.
- Come up with the parameters to evaluate the solution on. Examples 
  could be feasibility, implementation cost, dollar cost, architectural 
  fit, simplicity/understandability, security, performance, and others. 
  Don't just take this list, come up with an appropriate list yourself.
- Come up with some design options.
- Consider pros/cons of each design option, tradeoffs, gotchas, and 
  evaluate it against each of the previously selected evaluation 
  parameters. Critique each solution.
- Try to improve each design option based on the critique.
- Do a final review/ranking/evaluation of the design options and select 
  the best one.
- Output the candidates, the decision criteria, the evaluation of each 
  candidate against the criteria, and the final recommendation.

Do not make any changes unless explicitly prompted; just output your findings in chat.

The prompt might look something like this:

/tech-design a solution to this open question from my-spec.md:

>Question details…

Or this:

/tech-design the best approach to implement (whatever thing)

You can find additional context in (list of files)

Note that this skill doesn’t mention anything about modifying the spec document; in fact, it explicitly asks not to make changes. This is because I prefer to review the decision and, if I’m not happy with the result, re-prompt with additional information or ask follow-up questions and refine the design.

When I’m happy with the result, I explicitly ask:

Update my-spec-file.md to reflect the recommended approach.

Step 4: Review the designs

Aside from the usual human review, I also like to run this variant of the review prompt a couple times before settling on the design:

We’re about to start implementing my-spec-file.md. Before doing that, I’d like to do a review of the technical design. Do the following:

* Review the file and any relevant documents and code. Think through how you’d implement it, in detail. Do you have any concerns? Are there any gaps in the requirements or design? Any conflicts? Any remaining significant decisions to be made? Any unexpected interactions with existing functionality? Emit a list of issues.

* Review each issue from the previous step. For every one that is actually a non-issue, ignore it. For any issue that has a single obvious best answer or any that can be answered with “just do what the code does now”, update my-spec-file.md to reflect that. For all others, add them to the “Open questions” section.

This usually produces a number of edits and follow-up questions, which I review, correct, and answer. Sometimes I can answer the open questions immediately, and sometimes I invoke the /tech-design skill to answer them.

Sometimes these reviews reveal new missed requirements or unexpected interactions that force me to step back and rethink the approach.

I usually do a couple of these review passes before moving on to the next step.

Step 5: Phasing

At this point, the design is largely complete and now we’re moving on to the implementation.

For simpler features, we can, again, just tell the agent to plan and implement. However, for larger features I find it useful to break up the work into implementation phases.

This can take a few shapes:

• For features spanning multiple repos, I can ask the agent to figure out which PRs should be created in what order. Some models can be overzealous in splitting the PRs, so I might ask to bundle the changes in as few PRs as practically possible.

• For a large implementation in a single repo, I ask to break up the changes into phases containing maybe 2,000-5,000 lines of code each. This (1) helps me break down the changes into reviewable chunks, and (2) lets me catch issues early so that I don’t have to fix the whole 30,000 LOC changeset.

That’s about it

From here, I’m mostly in cruise-control mode: prompt to plan, prompt to implement, review, apply fixes, prompt to plan the next phase, etc.

I have other skills for generating plans the way I like, implementing the changes, reviewing the emitted code, and making PRs, but they’re outside the scope of this post.

I’m curious: what is everyone else doing? How are you handling requirements, design or planning?

                                  2016          2024
Population > 18               29,027,447    32,600,713
Units available               15,507,364    16,786,020
Units/person                       0.534         0.515
Units needed to get to
2016 levels                           -       630,000

I.e. we need 630,000 houses or apartments just to get back to 2016 levels of housing availability. A bit over half of this number comes from Ontario; repeating the math for that province gives us a shortage of 333,000 units compared to 2016.

To be clear, this number misses a lot of nuances: we don’t know which of these “units” are 8-bedroom houses and which are micro-condos, and we don’t know which of these adults are single and which have families.

On the surface, this shortage of 630,000 units doesn’t seem too horrifying: in 2023, the construction industry has started building 240,267 new units (CMHC housing starts table) — down from the peak of 271,198 in 2021 — so it might look like it should take a bit less than three years to catch up.

But remember: Canada is a growing country, so as we’re building these housing units, new people over 18 are appearing. So let’s update our math. Suppose Canada’s growth follows StatCan’s moderate growth projections (M2 scenario in this table), and we continue building houses at the same rate as we did in 2021 (the best year we’ve seen so far). How long will it take to catch up to the 2016 housing level?

Well, then the projection looks something like this:

Year    Population > 18    Units short
2025        33,283,800         577,130
2026        33,548,800         495,079
2027        33,822,800         418,454
2028        34,312,900         457,893
2029        34,800,100         496,398
2030        35,280,800         532,044
2031        35,761,400         568,248
2032        36,241,000         604,527
2033        36,715,500         638,690
2034        37,187,400         672,071
2035        37,652,100         702,210

Uh-oh, we never catch up.

So what will it take?

I’ll spare you the math and go straight to the answer: if we want to get back to 2016 housing stock levels by 2035, then over the next five years we need to increase the housing starts by 26% over the 2021 levels to about 380,000 starts/year, with the bulk of this increase happening in Ontario, which needs to raise its housing starts by ~45%.

It’s not a small amount, but not impossible either. For comparison, in the years between 2016 and 2021, housing starts have increased 37% Canada-wide ans 33% in Ontario.

But there are a few problems in the way

A lot of the discourse online and among activists has been focused on onerous regulations, slow approvals, and unhelpful building codes — and rightfully so. That said, these seem to be improving as the public mood shifts towards building more housing, and, looking at the industry reports, it seems like at least some of the insiders no longer see the public policy as the biggest long-term drag on construction.

There are a couple of other major problems though.

First, we need the new housing starts to go up, but instead since 2021, they’ve been going down. The explanation for this is fairly straightforward: less money has been spent on building new houses because the interest rates have gone up and house purchases have slowed down.

This problem will partly fix itself as the Bank of Canada lowers its policy interest rate. However, it won’t cut the rate back down to 0.25%, and the total investment into new housing isn’t projected to go back up to the 2021 levels either (source: BuildForce National Summary for 2024):

But we need the investment to go much higher: if the average historic costs per housing unit stay the same, we’ll need to be investing as much as $54 billion per year more than we’re forecast to catch up to our 2016 housing availability target. If that money doesn’t materialize from somewhere, no amount of regulation reform will help plug the gap.

The second problem is the shortage of skilled labor in the construction industry, which is projected to worsen due to retirement. In the short-term, construction is limited by lack of people with a few specific skill sets like crane operators, elevator mechanics, or ironworkers, especially in major cities like Toronto (source: BuildForce Ontario report 2024):

In the long-term, from the same report, based on the current hiring levels and trades education pipeline, Ontario alone is expected to be short 35,500 construction workers by 2033 — and that’s assuming BuildForce’s own not-quite-adequate construction investment forecast seen in the earlier graph. To get back to 2016 affordability levels in a reasonable time, we’d need to find more skilled construction workers somewhere, either by attracting more young people to trades locally, or through immigration.

Or… what if we lower the population growth?

While we work on addressing the supply-side issues of housing, there’s one lever in the hands of the federal government that can easily control the demand side: immigration levels.

How will the changes to the immigration policy affect the outcomes?

Let’s tweak some numbers and redo the math:

• StatCan’s M2 growth projection scenario already assumes a net departure of around 212k non-permanent residents per year between 2025 and 2027. That’s roughly consistent with reducing foreign students to about 2019 numbers and losing about half of the temporary foreign worker permit holders. Let’s keep this as-is.

• The current permanent resident admission target is 500k/year. Let’s lower it to 350k until 2035, which will still keep it mostly above the historic levels and at around 0.8% of the current population.

Let’s keep the construction industry output at their 2021 levels and see what the situation looks like:

Year    Population > 18    Units short
2025        33,283,800         668,723
2026        33,548,800         521,036
2027        33,822,800         382,890
2028        34,312,900         250,171
2029        34,800,100         233,516
2030        35,280,800         215,927
2031        35,761,400         195,478
2032        36,241,000         175,588
2033        36,715,500         155,773
2034        37,187,400         111,129
2035        37,652,100          85,173

We gradually catch up! Not quickly, but the effects should start being felt after the first year, and should improve every year after that. And if the construction industry raises its output by just 3%/year (for a total of 12% in 5 years), we get to 2016 housing availability levels by 2031.

Conclusions

It took a while to get into the current housing crisis, and it will likely take a while to get out. Whoever ends up taking on this will have their work cut out for them.

We’ll need:

• The federal government doing its part in controlling the demand by managing the immigration levels.

• Cities and provinces, especially Ontario and BC, to keep pushing for regulatory reforms;

• Provinces continuing to expand capacity for trades education and making it more attractive for young people;

• The construction industry finding ways to build better houses and apartments for less;

• Figure out how to keep funding flowing into the construction industry;

Whatever happens, I will keep following this with great interest.

But I can’t play like that. First, I’m lazy, and second, my slow human brain can’t do information theory calculations or depth-first searches.

Instead, every time I play Wordle, I enter the same sequence of words until it feels like there aren’t many solutions left, at which point I try to make more tailored guesses.

Here are a couple examples of my recent games:

Until today, my go-to words were STALE, CHIRP, MOUND, and GAWKY, picked mostly by gut feel while trying to check as many frequently used letters as possible. So far I’ve been doing alright with them.

But can we do better?

Yes! Let’s use math and programming to do better.

Though to do computer science, we’ll need a more precise algorithm than “play these words until it feels like there aren’t many solutions left”. Let’s go with this:

1. Pick four words.

2. In every game, keep guessing these words in order.

3. If there are 3 or fewer possible solutions left, make tailored guesses that minimize the number of turns to victory.

Oh, and let’s use only the list of 2309 valid solution words (heplfully provided in Wordle’s JavaScript source code), and ignore the larger list of valid guesses. In part, that’s to make the calculations simpler, and in part because you’ll forget that THEWY is an acceptable guess by the time you’ll finish reading this sentence.

Pick words that produce fewest remaining possible solutions?

That was the first thing I tried.

For the first word, I picked one that resulted in the lowest remaining information entropy (basically, fewest remaining options) on average across any of the 2309 valid solutions. That turned out to be RAISE.

Then, I picked the next word that gave me (on average, across all solutions) the most information after guessing RAISE. That turned out to be CLOTH. And so on, getting DUMPY and BEGAN as the best third and fourth words.

But how does it compare to the original STALE>CHIRP>MOUND>GAWKY? If we simulate all 2309 possible games we see that:

• RAISE/CLOTH/DUMPY/BEGAN wins all games and uses 3.89 guesses on average.

• My original STALE/CHIRP/MOUND/GAWKY wins all games and uses 3.84 (!) guesses on average.

😲 ⁉️ My original gut-feel word choices did better?

Because each word being optimal ≠ the full set is optimal

So it turns out it’s not enough to optimize each word; instead, we have to jointly optimize the entire sequence WORD1 WORD2 WORD3 WORD4 across all possible word sequences.

But that’s a lot of sequences! I’m not sure how much exactly 2309⁴ is, but it’s definitely more than my computer can handle. Even doing some intelligent pruning (e.g. ‘a letter must not repeat in the first 3 words’) still leaves a lot of options to evaluate. What else can we do to cut it down?

What I ended up doing was ranking all words by the number of bits of information that they provide, picking the top 10-100 words at each position for a total of ~160,000 four-word combinations, and then simulating all possible games for each sequence.

After two days at 100% CPU, my computer gave me the answer:

• SLANT PRICE DOUGH BALMY: 3.79991 average guesses to win.

But there were a lot of other sequences that produced similar results. For example #2 sequence was CRANE SPILT DOUGH BALMY that took on average 3.8008 guesses to win.

But what if I’m a smart and tailor my guesses sooner?

Or, to be less flippant, you can often get into situations where there are more than three guesses left, but you can easily enumerate them. The simplest example is if your first guess SHARE results in something like

There are lots of great videos and articles on the optimal Wordle strategy. I particularly enjoyed this YouTube video by 3Blue1Brown on using the information theory to come up with the best guesses.

But I can’t play like that. First, I’m lazy, and second, my slow human brain can’t do information theory calculations or depth-first searches.

Instead, every time I play Wordle, I enter the same sequence of words until it feels like there aren’t many solutions left, at which point I try to make more tailored guesses.

Here are a couple examples of my recent games:

Until today, my go-to words were STALE, CHIRP, MOUND, and GAWKY, picked mostly by gut feel while trying to check as many frequently used letters as possible. So far I’ve been doing alright with them.

But can we do better?

Yes! Let’s use math and programming to do better.

Though to do computer science, we’ll need a more precise algorithm than “play these words until it feels like there aren’t many solutions left”. Let’s go with this:

1. Pick four words.

2. In every game, keep guessing these words in order.

3. If there are 3 or fewer possible solutions left, make tailored guesses that minimize the number of turns to victory.

Oh, and let’s use only the list of 2309 valid solution words (heplfully provided in Wordle’s JavaScript source code), and ignore the larger list of valid guesses. In part, that’s to make the calculations simpler, and in part because you’ll forget that THEWY is an acceptable guess by the time you’ll finish reading this sentence.

Pick words that produce fewest remaining possible solutions?

That was the first thing I tried.

For the first word, I picked one that resulted in the lowest remaining information entropy (basically, fewest remaining options) on average across any of the 2309 valid solutions. That turned out to be RAISE.

Then, I picked the next word that gave me (on average, across all solutions) the most information after guessing RAISE. That turned out to be CLOTH. And so on, getting DUMPY and BEGAN as the best third and fourth words.

(Along the way, I had to keep in mind that I had to switch to “smart” guesses if I had 3 or fewer valid solutions left.)

Here’s what each word contributed:

Guess Ave. remaining entropy (bits) Ave. options remaining Worst case options remaining RAISE 5.2 17.5 167 CLOTH 1.8 2.38 30 DUMPY 0.8 1.53 6 BEGAN 0.4 1.28 4

But how does it compare to the original STALE>CHIRP>MOUND>GAWKY? If we simulate all 2309 possible games we see that:

• RAISE CLOTH DUMPY BEGAN wins all games and uses 3.89 guesses on average.

• My original STALE CHIRP MOUND GAWKY wins all games and uses 3.84 (!) guesses on average.

😲 ⁉️ My original gut-feel word choices did better?

Because each word being optimal ≠ the full set is optimal

So it turns out it’s not enough to optimize each word; instead, we have to jointly optimize the entire sequence WORD1 WORD2 WORD3 WORD4 across all possible word sequences.

But that’s a lot of sequences! I’m not sure how much exactly 2309⁴ is, but it’s definitely more than my computer can handle. Even doing some intelligent pruning (e.g. ‘a letter must not repeat in the first 3 words’) still leaves a lot of options to evaluate. What else can we do to cut it down?

What I ended up doing was ranking all words by the number of bits of information that they provide, picking the top 10-100 words at each position for a total of ~160,000 four-word combinations, and then simulating all possible games for each sequence.

After two days at 100% CPU, my computer gave me the answer:

• SLANT PRICE DOUGH BALMY: 3.79991 average guesses to win.

But there were a lot of other sequences that produced similar results. For example #2 sequence was CRANE SPILT DOUGH BALMY that took on average 3.8008 guesses to win.

But what if I’m a smart and tailor my guesses sooner?

Or, to be less flippant, you can often get into situations where there are more than three guesses left, but you can easily enumerate them. The simplest example is if your first guess SHARE results in something like

So let’s say that we start making tailored guesses when there are five possible solutions left instead of three. How does this change our optimal sequence?

After another day at 100% CPU, I ended up with TRICE SALON DUMPY GLOBE, which solved the puzzle in 3.72 steps on average.

But SLANT PRICE DOUGH BALMY wasn’t far behind, at 3.74 average guesses, and even the original STALE CHIRP MOUND GAWKY clocked in at respectable 3.79 guesses.

But what if the word list changes?

That’s a good question. It wouldn’t do to have a set of words that’s hyper-optimized for Wordle but not other similar puzzles.

So I took the list of words ranked by their occurrence in Google Books, and selected top 3000 five-letter words.

Redoing the same analysis here gives us a new winner: SAINT BORED CLUMP FIGHT at 3.91 guesses to win, though again, SLANT PRICE DOUGH BALMY is not far behind with 3.96 guesses (athough with 99.93% win rate), and even my original STALE CHIRP MOUND GAWKY comes in at a not-terrible 3.98 average guesses and 99.90% win rate.

So what’s the lesson?

So it turns out that the most optimal sequence really depends on how you play the game and which Worlde-like game you play.

But if you want a good-enough sequence, it looks like you can pick any three words that cover the most frequently used letters, and then a fourth word that complements the other three well.

Personally, I think I’ll try SLANT PRICE DOUGH BALMY for the next while.

So let’s say that we start making tailored guesses when there are five possible solutions left instead of three. How does this change our optimal sequence?

After another day at 100% CPU, I ended up with TRICE SALON DUMPY GLOBE, which solved the puzzle in 3.72 steps on average.

But SLANT PRICE DOUGH BALMY wasn’t far behind, at 3.74 average guesses, and even the original STALE CHIRP MOUND GAWKY clocked in at respectable 3.79 guesses.

But what if the word list changes?

That’s a good question. It wouldn’t do to have a set of words that’s hyper-optimized for Wordle but not other similar puzzles.

So I took the list of words ranked by their occurrence in Google Books, and selected top 3000 five-letter words.

Redoing the same analysis here gives us a new winner: SAINT BORED CLUMP FIGHT at 3.91 guesses to win, though again, SLANT PRICE DOUGH BALMY is not far behind with 3.96 guesses (athough with 99.93% win rate), and even my original STALE CHIRP MOUND GAWKY comes in at a not-terrible 3.98 average guesses and 99.90% win rate.

So what’s the lesson?

So it turns out that the most optimal sequence really depends on how you play the game and which Worlde-like game you play.

But if you want a good-enough sequence, it looks like you can pick any three words that cover the most frequently used letters, and then a fourth word that complements the other three well.

Personally, I think I’ll try SLANT PRICE DOUGH BALMY for the next while.

So here are some of the less obvious REST API design lessons that I’ve learned over the years.

1. Do not nest objects in URL paths

Suppose you have task objects that always belong to project objects. What would should be the URL to get information about a task? Should it be this:

GET /api/projects/{projectId}/tasks/{taskId}

or this?

GET /api/tasks/{taskId}

Having gone both ways, I now confidently think that the bottom choice is the best one. If you have a taskId, then projectId is redundant, and by including projectId in the URL you’re making the caller look up a piece of information that they may not have or need otherwise. And what happens when the caller provides the wrong projectId? Should you return 404 Not Found even though you can unambiguously locate the task? You’ll also have to add extra validation code to check that the task is really in the right project—something that you wouldn’t need to do with the shorter version of the URL.

You may ask: “But how do you add a task to a project then?”

Well, how about this?

POST /api/tasks
...
{
    “projectId”: 1234567,
    ...
}

You may then ask, “But how do you look up tasks for a project if you can’t do GET /api/project/{id}/tasks?”

Like this:

GET /api/tasks?projectId=<projectId>

As a bonus, you get an endpoint to list all the tasks irrespective of which project they’re in. And this approach treats projectId as one of multiple optional filters that may be applied on /api/tasks, alongside maybe status, assignedTo, or any others.

2. Of the 2xx HTTP status codes, you only need 200 OK

This one wasn’t obvious to me until a coworker recently pointed out that he’s never seen a case where the caller had to make a decision based on the type of 2xx return code they got back.

And indeed, although RFC 9116 — the current HTTP spec — defines seven different 2xx success codes, in practice, the client-side code almost never needs to make a decision based on whether it receives e.g. 200 OK or 201 Created. Alright, in very rare cases it’s convenient when the server returns 204 No Content when you’re debugging and trying whether the request is really supposed to be missing a body. But you can accomplish the same thing with Content-Length header, or by some other means.

I have, however, seen developers spend hours going back and forth on which 2xx code should be used for a specific endpoint. So to save everyone’s time, let’s just use 200 OK and forget about the rest.

3. Encode timestamps in a human-readable format

If you’re sending or receiving timestamps, it may be tempting to use something like the Unix epoch time in milliseconds (the number of milliseconds since 00:00 UTC on Jan 1, 1970). It’s not a bad choice, and it’s quite convenient to work with.

Except it’s not human readable. Something like ISO 8601 (e.g. 2023‐07‐12T23:57:23Z) is easier to read for a human, and you can even drop the colons (2023‐07‐12T235723Z) to make it URL-compatible.

It may take a bit extra work to configure your framework to emit times in the right format, but it will make using (and debugging) the API a lot easier.

By the way, don’t forget to include time zone in your timestamps, even if it’s just a Z at the end.

4. Pagination: avoid offsets and provide a link to the next page

There are broadly two ways to do pagination:

• Page number and page size: e.g. https://.../things?page=10&pageSize=20. This will work alright up to maybe page 10,000 or so, at which point, if the server is fetching the results from a database, you’ll start running into a quirk with OFFSET that can substantially degrade performance. So while this pagination approach is convenient when the results go into a paginated UI, it’s a bad idea to use it for an endpoint that will be called by export jobs or by anything else that will ask for a large number of pages with high offsets. Instead, it’s much better to use…

• WHERE-based pagination: where the caller provides something akin to a where clause, like https://.../things?fromId=12345678. If this argument corresponds to a DB index, the response times should stay constant-ish as fromId argument increases, unlike OFFSET whose response times tend to increase linearly.

Oh, and if anyone will be iterating over multiple pages of responses, it really helps to include the URL of the next page in the response body:

{
    "data": [ ... ],
    "metadata": {
        "nextPage": "https://..."
    }
}

That’s something that I’ve been curious about for a while now, so I’ve read a bunch of papers on the subject.

Metal electrodes

Surprisingly, off-the-shelf 44 AWG insulated copper wire is not a completely terrible choice. Its 51μm diameter (plus insulation) is a bit on the high end for a typical intracortical electrode, but it’s about the right order of magnitude. If you stick it in, for a short time it has a decent chance of picking up electric spikes that nearby neurons send when they get excited.

But it’s still a terrible choice. First, the copper will quickly oxidize and corrode in the brain’s electrolyte-rich fluid. And second, copper is toxic to the neural tissue. So don’t put copper electrodes into anyone you love.

Okay, let’s try something else. How about an off-the-shelf stainless steel 40μm wire? These are actually a legit choice, once you figure out how to insulate and sharpen them. In 1958, at UCLA, Felix Strumwasser sharpened 80μm stainless steel wires with hydrochloric acid and used them to record spikes in neurons of ground squirrels for over a week. The wires were likely insulated by dipping them into lacquer, as described by Green, also from UCLA, later the same year.

However, the most popular electrode metal these days seems to be tungsten or platinum. Though noticeably pricier, it seems that these give you a better signal-to-noise ratio without messing with the brain chemistry.

So let’s say we get some platinum or wire, coat it with tissue-friendly parylene C for insulation, and then expose and sharpen its tip. Are we good?

Not quite.

Multiple electrodes

According to Techniques for Large-Scale Multiunit Recording, inserting one electrode by itself is often not that useful. First, in addition to neural signals, the electrode will also pick up the background noise, including the AC power lines in the walls. Second, we often want to collect spikes from multiple sites to observe the behavior of more neurons.

The simplest way to increase the number of electrodes is to twist a few of them together, insert them, and measure the difference in voltage between some of the pairs. Taking the difference in voltage between the electrodes helps not only remove the background noise but also isolate individual neurons because the electrodes closer to the spiking neuron will register higher voltage swings.

The next step up is electrode arrays, which help measure signals over larger areas or a range of depths. Here, silicon probes became popular, like the widely used Utah array with as many as 100 needle-like electrodes, or a Neuropixels probe, which has a single shank with 960 electrodes.

Utah array in particular featured in a number of really cool experiments, like this BrainGate experiment where researchers implanted Utah arrays into a tetraplegic individual and connected them to electrodes implanted in the hand muscles, allowing the subject to move his hands.

However, it turns out that silicon probes have their own problems, and aren’t the best thing to insert into the brain either.

Biocompatibility

So what’s the problem with the silicon probes?

Well, a number of studies have shown that over the months after the implantation, the signal from the electrodes weakens, and some of the previously working electrodes go silent.

And metal electrodes aren’t better either. For example, this study tracked the performance of 50μm tungsten electrodes over 9 months, and found that within the first few months, many electrodes could no longer detect spikes and the few that could, had much weaker or unstable signals. In some individual rats (who got the arrays implanted), by the end of the experiment, zero out of sixteen electrodes could detect useful signals

To be clear, all of these studies showed that in almost all cases it was still possible to get useful signals at the end of the experiment, just that over time the signal quality got noticeably worse.

So can we do better?

A lot of studies have dug into why the signal quality decreases. This recent review (section 2) summarizes the causes: (1) the electrodes create small stab wounds on the way in, damaging the tissue and the blood vessels; (2) electrode bodies cause chronic inflammation that results in a scar that surrounds the probe and reduces the signal quality; (3) all of the above cause nearby neurons to die or migrate away from the insertion site.

This can be made worse by the rigid bodies of the probes. Mammalian brains naturally move inside the skull, swelling with each pulse of blood from the heart, and rigid metal and silicon electrodes cannot deform at the same rate as the brain tissue.

To top it off, poorly chosen insulation can crack or detach from the probe, letting carefully collected signals leak back into the surrounding tissue.

Smaller, softer electrodes

Various recent reviews, such as this one and this one, suggest that the best electrodes would be:

• Made of very thin wires, comparable in size to neuron’s appendages (axons and dendrites) - that’s <10μm across.

• Soft, bending as easily as cortical tissue.

• Covered in well-attached insulation that won’t agitate neurons or other cells in the brain.

• “Floating”, that is, loosely attached to the skull or signal processing circuitry, allowing the electrodes to move as the brain moves.

Over the last decade, a number of electrodes satisfying these conditions started appearing:

• NeuroRoots, made of 100nm-thick platinum wires encapsulated in parylene-C insulation; total cross-section: 1.5μm x 7μm.

• Elon Musk-sponsored Neuralink, which uses insulated gold “threads” that are 4-6µm-thick and 5-50µm wide, depending on the number of electrodes per thread.

• Neurotassels uses 100nm-thick gold electrodes wrapped in polyimide insulation; total cross-section: 3µm x 12µm.

• Ultra-soft conducting polymer microelectrodes, though at 125µm these were much thicker than others on this list.

• Carbon fiber electrodes around 9-10µm in diameter.

• And a variety of other wires, meshes, and probes.

Most of these came with a study showing how much they outperform the previous generation of electrodes, and how little damage they do in a rat brain. In the case of something like NeuroRoots, it looks pretty impressive on paper: after 7 weeks, all 32 of the 32 implanted electrodes could pick up signals nearly as well as on the day they were implanted. In the case of the conducting polymer above, the study compared it to much thicker tungsten electrodes, making the comparison somewhat unfair.

Conclusions

So basically, yes, you would need a brain-grade wire to collect signals from neurons, and we’re still working out the specifics of what makes the wire “brain-grade”.

With the progress above made in the 2010s, I’m looking forward to seeing what the 2020s will bring.

The FPGAs are certainly… different from any other kind of programming, and it took a bit of effort and troubleshooting to get my FPGA calculator working. If you’re interested in getting into FPGAs, here are a few tips below.

1. The basics

Programming FPGAs is different from programming in JavaScript, C#, or C. There are no levels of abstraction between you and the hardware, so it’s essential to learn the foundations of digital logic.

For me, The Art of Electronics worked very well — specifically, chapters 10-14.

The book first introduces the basic logic circuit elements, and then builds up to other components and concepts: flip-flops, clocking, different logic families, logic interfacing, three-state outputs, busses, etc.

Almost all of it was useful when I got to programming the FPGA.

If textbooks are not your thing, there are lots of YouTube videos covering the same subjects.

2. Verilog

There are three main hardware definition languages: Verilog, VHDL, and SystemVerilog. Either one is a valid choice, though I found that for Digilent and Xilinx (my choice of FPGA boards and chips) there were more Verilog examples, so I went with that.

Unfortunately, Verilog just doesn’t have as much great documentation as, say, TypeScript or C#. The best I could find were these two websites:

• Chipverify Verilog Tutorial

• This Verilog guide on readthedocs.io.

I’ve read these before I started programming, and referred back to them a number of times afterwards.

3. The board

You can try buying an FPGA chip and soldering it to a board yourself, but it’s much easier to get a starter board that already includes all the necessary components such as configuration flash memory, USB interface, and voltage regulator.

For starter boards, I liked Digilent boards the most and went with Cmod A7 that had a Xilinx Artix-7 chip on it. It was powerful enough to fit the projects I wanted to build with room to spare, and it could plug into a breadboard.

Alternatively, if you’re looking to spend less, Cmod S7 seems like a decent choice, and if you’re looking for more power, there’s something like Arty A7.

4. Making the LED blink

Now we have everything we need to take FPGA for a test drive.

If you’ve gone with a Digilent board, the next step that worked for me was to go through the Getting started with Vivado guide from Digilent.

Be aware that there are a few quirks that you may run into:

• The LED port name in the module definition must exactly match the port name in the constraints file. For example, it can’t be led[0] in the constraints file and led in the module definitions - you’ll get an error when generating the bitstream.

• Xilinx wants you to create an account before using Vivado, and they want you to enter something that looks like a real address when doing so. They’ll check the address, but not too thoroughly. You’ll need to log into the account when starting Vivado.

5. Now build your first project.

From here, build whatever you were thinking of.

If you’re looking for an idea for a simple project before moving on to something more interesting, I found a calculator to have just the right amount of complexity:

From here on, the things that helped me the most were:

• Carefully reading the documentation and the guides, as they often contain the answers you’re looking for. For example, each Digilent board has a detailed reference manual that explains how to work with each component that’s on the board.

• The chip manufacturer has detailed guides as well. For example, if you’re looking to change the clock frequency on your FPGA chip, or want to work with multiple clock frequencies, Xilinx has a very thorough guide on working with Mixed Mode Clock Managers.

• If you run into a problem, keep simplifying your logic until it starts working. For example, when I was having trouble sending data over the USB/UART link, I ended up removing all of my logic and just tying the RXD (send) wire to TXD (receive) wire to make sure that I could echo the characters through it using the COM terminal. Turns out I couldn’t because I mixed up which wire was the input and which was the output. The rest of my logic was correct.

• If you get stuck, like everywhere else, use Google. Someone has run into a similar problem. If you don’t find the exact solution, at least you may get an idea that you can try next.

6. This will save you days

I was going to lump this with the tips above, but this deserves its own section.

Put

`default_nettype none

at the beginning of every Verilog file (yes, including the backtick), and

`default_nettype wire

at the end.

This is because by default, if you make a typo in a variable name, Verilog will assume that you’ve defined a new variable of type wire. Because of this, I literally spent a day debugging a problem that was caused by a typo.

If you put `default_type none, Verilog will complain that it doesn’t know the variable type and will bring your attention to the typo.

The closing `default_nettype wire is necessary because apparently some of Xilinx’s own modules (called “IP” in FPGA terminology) rely on implicitly defined variable types.

So when we see something in front of us, like a pencil, how exactly does the brain understand that yep, that’s a pencil?

Like, what is the detailed sequence of steps that happens? The eyes presumably convert the light to neural signals (how?), then these get sent to the brain (where?), and then the brain does… what?

I’ve been reading through a bunch of neuroscience textbooks and papers for the past six months, and I think I’m at the point where I’ve now got the whole picture.

So here are the core pieces of this picture, with a lot of details omitted to fit it into a short blog post.

Let’s start with the input.

What exactly do the eyes do?

The main output of the eyes are the neurons called ganglion cells that live in the thin layer of tissue at the back of the eye. To grossly oversimplify, these ganglion cells transmit the brightness of each pixel of the picture that’s projected onto the back of the eye, though the reality is much more complex.

To transmit this information, the neurons use spikes of voltage – that’s when the voltage between the inside and the outside of the cell suddenly rises from somewhere around -60 mV to around 0 mV and falls back down over the course of about a millisecond.

In the eyes, these spikes are caused by light-sensitive proteins called opsins that live in cells of another type, the photoreceptors. When photons hit the photoreceptor cells, the opsins in these cells absorb the photons and slightly change their structure, which causes a rapid cascade of chemical reactions across a few cells. At the end of this series of reactions, the ganglion cells let in some positively charged ions, raise their internal voltage, and start a spike.

(I’ve written more about how the ions cause spikes in another post)

The whole system is set up to handle different volumes of light: the more light is shining on the photoreceptors, the more spikes are produced by the ganglion cells.

So, to grossly oversimplify even more, more light => more spikes in the ganglion cells.

And now what?

Where do the signals go?

The ganglion cells live at the back of the eyes, but they have long output extensions (axons) that run from the eye, through the optic nerve, and right into the center of the brain. That’s where the thalamus – the brain’s main switchboard – is located.

In an adult human, that’s about 10 cm or 4 in of length, and that’s far from the record for the length of a neuron. The spikes travel along the full length.

One of the interesting things that happen along the way is that the optic nerves cross at about the half-way mark, and the neurons from the left half of each eye end up traveling to the left half of the brain, while the neurons from the right half of each eye go to the right.

This way, each hemisphere of the brain will be able to construct a stereo picture of each half of the visual field.

From the thalamus, a new set of neurons picks up the same information and forwards it to the back of the brain - the occipital lobe - where the brain will do its magic.

And the brain does… what?

In the brain, visual processing is done through a sequence of visual cortex areas - V1, V2, V3, etc, each of which extracts progressively more complex features from the original signal.

The first one of these, V1, is located right at the back of the brain; it represents nearly pixel-for-pixel what the eye sees - that is, the neighboring neurons in V1 represent neighboring points on the image.

In V1, a series of neural connections detect basic features like edges, colors, directions, or motion in each patch of the visual field. If the neurons responsible for detecting a specific feature detect it, they fire a series of spikes, letting the downstream areas know what they found.

The downstream areas - V2, V3, etc. - extract progressively more and more complex features over larger and larger areas.

If you’re familiar with Convolutional Neural Networks (if you’re not, here’s a good explainer), that’s pretty much exactly what’s going on here.

Early on, the processing path splits into two branches: the bottom one that’s focused on recognizing objects, and the top one that’s focused on building a mental picture of space and motion.

At the end of the bottom branch are the areas that recognize faces, places, or objects, including the pencil that we started with.

Here, if the final collection of features extracted along the way add up to a specific known object, a collection of neurons that are associated with that object fires spikes to other areas of the brain, triggering appropriate emotions, thoughts, memories, and other associations that we may need at the moment.

What’s interesting about it are both its simple wiring (at first glance), and the extensive hidden support circuitry that can tell us more about how the brain works.

VOR is not the only circuit that helps us keep our eyes locked on target; there are others that work in parallel. I suspect that they probably use the same underlying principles.

(Note: some links here are behind a paywall. Remember that it’s illegal to go to Sci-Hub and search for the DOI link of the article to get free access to it.)

So what’s Vestibulo-Ocular Reflex?

Pick a point on the screen, and while looking at it, turn your head left to right. Your head will move, but your eyes will stay locked onto that point.

For that to happen, the muscles that control your eyeballs need to change their tension at exactly the right time and by exactly the right amount to compensate for the movement of the head.

While at first glance that may seem tricky, in reality, the brain deals with it the same way as the smartphone: it just uses our internal accelerometer, the vestibular apparatus, pictured below in the red box.

From accelerometer to muscles

From the vestibular organ, there’s a simple three-neuron-long connection that feeds the accelerometer signal to the muscles that move the eyes, first documented in the 1930s. We can follow it to see what it does.

The neurons coming out of the vestibular organ output a steady stream of electric spikes. When the head turns left, the left vestibular neuron starts producing more electric spikes, and the right one - less. When the head stops moving, both vestibular neurons go back to the equal rate of spikes. If the head turns right, the opposite happens.

The signals from the vestibular neurons end up in the brainstem, the more basic part of the brain that sits between the cerebral cortex and the spine. There, another set of neurons directly connect them to the pools of motor neurons that move the eye muscles.

The muscle neurons, in turn, work according to rules we won’t go into right now, but it’s safe to simplify and say that the more excited they get the more the eye muscles contract.

So basically, when the head turns left, the increased rate of spikes from the left vestibular neuron is directly fed into the motorneurons that turn the eyes right.

End-to-end, this circuit is very fast. Experiments show that once the head starts turning, it takes less than 10 ms for the muscles to engage.

(Btw, Wikipedia has a different wiring diagram; I’ve seen it drawn both ways in papers. In practice, the result is the same either way.)

Complication #1: the neural integrator

So, on the surface, it seems that we’re done. The acceleration output feeds directly into the motorneurons, the eyes move, and we’re happy.

But here’s the problem: when the head is done turning, the vestibular organs go back to signaling at the base rate. Then the tension in the eye muscles goes back to equal, pointing the eyes straight. That’s not what we were hoping to see.

To fix that, we need to add a neural integrator to the picture.

The neural integrator does this: (1) it collects the total velocity signal sent to the eye muscles, and (2) outputs the spikes at exactly the right rate to keep the eyes pointed in the right direction.

I.e. it literally integrates the angular velocity to come up with the final position.

Our best guess that the integrator works using some clever mutually inhibitory feedback. If the left vestibular neuron sends more spikes, the right half of the neural integrator gets more excited, and ends up inhibiting the left half more, which ends up inhibiting the right half less. This entry in the Encyclopedia of Neuroscience describes the math in more detail.

Some papers indicate that there may be a number of different neurons involved, and this may be a multi-stage process.

(On a tangent: there isn’t just one neural integrator. There seem to be a number of them in the brain; for example, if you close your eyes and move around the room, a part of the cerebral cortex will keep an estimate of your position from your motion, though it won’t be very accurate. One of the papers linked above even argues that accumulating the evidence for two alternatives in a decision process could also be considered an integrator.)

Complication #2: controlling the gain

Setting the neural integrator aside, here’s a different question: why is it that a 5° turn of the head causes a 5° turn of the eyes? What makes it not be 4°, or 10°? Are we just lucky that the neural connections work out to be exactly right?

Turns out, there’s a circuit that runs through cerebellum that keeps the gain (ratio of head turn to eye turn) at exactly the right value. It’s a pretty clever circuit that turns out to be so useful that the brain has copy-pasted it thousands of times (with a few changes) and reused it in nearly all motor functions, plus a number of non-motor functions.

The center of this circuit are the inhibitory Purkinje cells, which, in this case, output a low, constant stream of spikes that lightly inhibit the connections from the vestibular neurons to the motorneurons.

The vestibular signal comes in through excitatory parallel fibers that make connections with Purkinje cells. When the head turns, these fibers make the Purkinje cell a bit more excited, making it inhibit the VOR circuit a bit more, kinda like tapping on the break.

Most of the time, this inhibition is just right to make the 5° head turn result in 5° eye turn. But what if it’s wrong?

This is where another set of neurons comes in - the climbing fibers. When the brain detects that the image on the retina lags behind motion (i.e. the inhibition is too strong), these fire. This sets the Purkinje cell ringing like a bell for a bit, and it turns off some of the connections from the parallel fibers, reducing the inhibition, and letting the eyes move faster.

I haven’t seen a good description for how the opposite error is fixed (eyes move too fast), but from what I remember from other circuits, the parallel-to-Purkinje connections gradually turn back on when the parallel fibers keep firing, so the error may just fix itself over time.

That’s it

That’s basically all I know about the Vestibulo-Occular Reflex so far. As I mentioned, there are other mechanisms to keep the eyes steady, like the neck reflexes, or the slower and finer Opto-Kinetic Reflex that uses the information from the retina instead of the vestibular organ.

There’s also a YouTube version of this article.

The picture above shows some (far from everything!) of what we know about the cells and neural circuits in the human retina.

What’s interesting is - the retina is not just a light detection organ. It’s a complex electro-chemical calculator with three neural layers, and it uses these layers to extract a number of useful signals from the light that hits our eyes.

Below, we’ll step through some of the more interesting features of the retina, but if you’d like to learn more, the above picture (and this article) are mostly based on this review article published in Feb 2020 in the journal Progress in Retinal and Eye Research, and on chapter 17 of the latest edition of Ryan’s Retina.

Photoreceptors are always on - except when there’s light

The photoreceptor cells in the vertebrate retina - rods and cones - are at the very back of the eye, behind the other layers of neurons.

Cones work best in bright light and can detect colors. In humans, there are three types of cone cells, each specializing in detecting red, green, and blue light. Rods work best in the dark, but can’t detect colors - just the brightness level.

Since the photoreceptors are at the very back of the retina, light has to navigate a tangled maze of neuronal bodies and wires to get to them. It does so using something called Müller cells (not shown), that stretch across the entire thickness of the retina and act like fiberoptic cables.

When the light gets to the photoreceptors, it turns them down. In complete darkness, the photoreceptors continuously emit neurotransmitters (“fire”, or “are excited”) at their maximum rate. The light inhibits them.

Parallel channels for brightness and darkness levels

For red and green cones, right from the start, the signal splits into two separate channels: ON cells, which fire neurotransmitters when the light is shining on their cone, and OFF cells that fire when the light is not shining on their cone.

Since the cones fire most when it’s dark, we can say that they produce the OFF (darkness) signal. So how do the ON cells fire when there’s light? Well, the ON bipolar cells are intrinsically excited and want to fire neurotransmitters, but they get inhibited by the photoreceptors. So as long as the photoreceptors are firing, ON bipolar cells aren’t. When there’s light (and photoreceptors stop firing), ON bipolar cells are no longer inhibited, and produce more neurotransmitters.

Why are there two channels instead of just one? Good question. Maybe so that it’s easier to recognize dark objects on light backgrounds more easily?

Either way, as far as we can tell, this ON/OFF channel pair only occurs with the red and green cones. Blue cones and rods have only ON bipolar cells. Though with rods, the story is a bit more complicated.

Rod signals are carefully fed into cone pathways

Humans (and other primates) have a fantastic vision. One of its crazy feats is that it can see across 10 log units of light intensity by using different light detectors and different retinal circuits in different lighting conditions.

Rods can detect single photons (!) and become saturated when it gets brighter than dusk/dawn. Rod signals are further amplified by neurons that collect multiple rod signals, suppress the noise, and feed the signals into the cone signal pathways.

This is done with help of the so-called AII amacrine cells (that’s roman numeral “2”, so it’s really “A2”, but the literature calls it “AII” for historical reasons). These AII cells collect inputs from the rod bipolar cells (which come only in ON variety), and use that signal to inhibit the cone OFF bipolar cells, as well as directly feed voltage into the ON bipolar cells through the connections called “gap junctions”.

This way, even when it’s dark, the same parts of the brain can be used to process the incoming information even though it’s been collected by a different set of photoreceptors.

Motion gets detected starting in the retina

Some output neurons - Parasol cells (ON and OFF varieties) and Recursive cells - produce the biggest response when the stimulus moves across its visual field.

What kind of neuron circuits make that happen?

For parasol cells, this paper suggests that it’s a network of special diffuse bipolar cells that makes it happen.

When the signal moves left to right in the image below, the bipolar cells excite each other through gap junctions that let voltage jump from one cell to the next.

Due to the internal mechanisms of neurons, these signals add more than linearly and produce a big response in the last bipolar neuron - ultimately exciting the parasol cell a lot.

Horizontal cells sharpen the picture

Retina also has a number of circuits for sharpening the image and enhancing the contrast.

One of them involves the so-called “horizontal cells” (yellow “H1” and “H2” cell in the image at the start of this post). These live in the same layer as the bipolar cells and connect to a lot of nearby cones.

The way the horizontal cells work is pretty clever: they are excited by the cones, and then, in turn, inhibit the cones back.

To see how these sharpen the image, let’s carefully look at what happens when the light shines only on one photoreceptor in the picture above.

If only one cone receives light, then

1. It fires less, so

2. Horizontal cell is less excited, and so it fires less too, therefore

3. Horizontal cell doesn’t inhibit the other surrounding cones as much, and so

4. Other cones fire more, producing a stronger OFF (darkness) signal.

The result is that we see a bright dot surrounded by a darker ring, making the dot more prominent.

By the way, this type of center-surround contrast enhancement is a common feature across all sensory systems, including touch, hearing, and smell.

Like image sharpening in Photoshop, this process can leave artifacts. The most famous one is below - the Hermann grid. The dark spots at the intersections disappear when you look straight at them because the surround inhibition fields are larger at the periphery of vision than in the center.

Blue cones are lower-class citizens in the retina.

For various reasons, blue cones get treated differently by the retina. They sit a bit farther back from green and red cones, they don’t get OFF channels, and there aren’t output (ganglion) cells that produce a pure blue signal.

Instead, the ganglion cells responsible for blue color combine blue ON signal together with red+green (= yellow) OFF signal to produce a “relatively blue” signal.

Blue cones are also sparser and aren’t present in the very center of the retina. This is in part because the lens in the eye produces chromatic aberrations at shorter wavelengths, and tighter spacing between blue cones wouldn’t help us see better.

Dopamine acts as additional gain control

Retina also adapts to light through Dopaminergic Amacrine Cells (DAC - dark blue on the diagram above). They spread their tentacles throughout the retina and around the bodies of some neurons.

When the light becomes bright enough, they ramp up the dopamine production, which in turn lowers the volume in the retinal circuits through various mecanisms.

Over the years I’ve come across a few approaches that help coordinate work between threads without having to make anyone wait. Here are a couple of them.

WARNING: I stopped using these patterns because they are easy to misunderstand and misuse, which can lead to random hard-to-diagnose bugs. If you use these patterns, you’re not helping the next develper understand the code better or avoid mistakes. You have been warned.

Reference to an immutable object

We know that in a multithreaded environment, modifying a shared object without locks is a bad idea. But what about replacing it?

For example, consider this code. I’ll use C#, but the same applies in Java, C++, or another traditional language with threads.

public class FunWithThreads {
    private Thing thing;

    public void InThreadOne() {
        this.thing = new Thing()
    }

    public Result InThreadTwo() {
        var localThing = this.thing;
        var result1 = SomeCalculation1(localThing);
        var result2 = SomeCalculation2(localThing);
        var finalResult = DoMoreWork(localThing, result1, result2);
        return result;
    }
}

Suppose lots of threads are concurrently calling InThreadOne(), while lots of other threads are concurrently calling InThreadTwo(). Will we ever get a wrong result, a race condition, or a conflict?

Not really. InThreadTwo() always refers to a consistent version of this.thing because it places the reference to it into a local variable, and from then on uses only the local reference. If the calculations take too long, then maybe we get a result that’s a bit stale, but that result will always be consistent.

Where it’s useful: I found this approach to be useful whenever this.thing is a large, pre-computed data structure that gets infrequently updated but frequently read. Mostly I’ve used it to cache some data for internal capital markets applications to make the application feel snappier to the user.

The pitfalls: It’s really important to make sure that this.thing is referenced only once at the beginning of the method, or else we’re back to race conditions. It’s also really important that this.thing is never modified after it’s assigned. That means lots of descriptive comments, very careful code reviews, and making sure that every developer that ever looks at this code knows all the gotchas. In a larger codebase, someone (maybe even you!) might still find a way to make a mistake somewhere.

Complex atomic operations

The reason the above code works is because in C# (and in most other languages) assignment of references is an atomic operation, i.e. while it’s executing, another thread can’t sneak in mid-way and do something else.

Modern processors have a few other interesting atomic operations, like compare and swap, implemented in C# as Interlocked.CompareExchange(), and in Java as AtomicInteger.compareAndSet(). This function takes three arguments and does the following as an atomic operation:

1. Compares the first and the third arguments, then

2. If they’re equal, sets the first argument to be equal to the second argument, or

3. If they’re not equal, does nothing.

One case where this can be used is when trying to limit requests for an expensive operation on a limited resource. For example, this code will drop any requests if the expensive operation is already running:

// Let’s pretend this class is a singleton.
public class ExpensiveOperationLimiter {
    private int isProcessing = 0; 

    public void RequestExpensiveOperation() {
        // If isProcessing == 1, this does nothing and returns 1.
        // If isProcessing == 0, this sets isProcessing to 1 and returns 0. 
        var wasProcessing = Interlocked.CompareExchange(ref isProcessing, 1, 0);

        if (!wasProcessing) {
            // Only one invocation will be running at a time.
            DoExpensiveOperation()
        }

        isProcessing = 0;
    }

    private void DoExpensiveOperation() {
        // ...
    }
}

If you look at the above code carefully, you’ll see that you can’t replace Interlocked.CompareExchange() and still have thread safety - there will always be an opportunity for another thread to sneak in between the instructions.

With a bit of work, this code can also be expanded to queue up the next request.

Where it’s useful: I’ve used this kind of code in situations where events can randomly trigger an application server’s cache to refresh from a database or from another system.

The pitfalls: Generally, I’d consider this to be a micro-optimization and not worth the effort. Many developers understand or have heard of locks, and have an easier time following them than any code that involves compare-and-set. As before, by using this, you’re risking someone (including you!) eventually making a mistake.

The best way to avoid locks

Don’t write multithreaded code!

One of those gaps was about how exactly the information travels through a neuron. Up to this point, all I knew was “something involving electric pulses”.

The reality, it turns out, is more interesting: neurons use ion channels to create and strengthen electric signals, and propagate information along their lengths.

So here is more or less how I understand this so far.

The setup - the neuron

First, a quick recap of the neuron structure.

A neuron has the main cell body. From that body, it grows a large number of branched extensions called “dendrites”. These usually serve as the input “wires”, though they also do some signal processing.

Neuron usually also grows another long extension called an “axon”. This is normally the cell’s output, and it can be up to a meter long in humans, though in the brain it’s usually shorter.

Let’s zoom in to the axon. First, we’ll look at shorter axons, because longer ones have myelin sheath, which will complicate the picture.

Ions provide the electric charge

The fluid inside the neuron (cytoplasm) and the intercellular fluid between the neurons have a number of ions dissolved in them: calcium (Ca²⁺), potassium (K⁺), sodium (Na⁺), chlorine (Cl⁻), and others.

For various reasons, neurons actively work to pump out Na⁺, Ca²⁺, and Cl⁻, so these are much more concentrated outside the cell than inside the cell. If the membrane wasn’t keeping them out, they’d quickly rush into the cell. Some still leak back in.

Similarly, neurons actively work to concentrate K⁺ inside.

Each of the ions carries a charge, so the end result of this pumping is that the cell’s inside is negatively charged relative to the outside. This voltage difference is called the “membrane potential”. Resting membrane potential is typically between -75 and -40 mV (millivolts).

Ion channels create voltage spikes

Ion channels are small valves set in the cell’s surface that let ions through if certain conditions are met. Some are opened by specific chemicals binding to them (e.g. neurotransmitters), while others open when the voltage inside the cell rises high enough. The latter are called “voltage-gated ion channels”, and they’re the ones we’re interested in.

Voltage-gated ion channels exist on all parts of a neuron, but are most concentrated in the axon. Most let through a specific ion: there are Na⁺ channels, K⁺ channels, Ca²⁺ channels, etc.

Let’s take a look at how Na⁺ channels and K⁺ channels would work to produce a voltage spike (called an “action potential”).

If something increases the cell membrane voltage to -55 mV, Na⁺ channels quickly open and let in a lot of Na⁺ ions. These ions carry a positive charge, so they quickly raise the voltage across the membrane to +30 mV. Then the channels automatically close. This happens over the span of milliseconds.

At the same time, K⁺ channels open and start letting out some K⁺ ions, opposing the Na⁺ current and slowing the voltage increase. The channels first open slowly, but after a while, the K⁺ current strengthens.

To start, Na+ channels dominate, but after they close, K⁺ outflow quickly makes the membrane voltage drop. This drop overshoots the initial resting level before leveling off at the voltage where K⁺ concentrations stay stable.

Then K⁺ channels gradually close.

The total number of ions exchanged throughout the whole process is minuscule, and has little impact on the overall concentration of ions inside or outside the cell.

Voltage spike opens nearby channels

When the channels create a voltage spike, they then trigger other nearby channels, which then trigger other channels, and so on. This makes the voltage spike travel down the length of the neuron.

The ion channels usually take time to reset before the next firing, so a single voltage spike doesn’t usually set off the entire neuron ringing indefinitely. However, the voltage can propagate from the base of the axon back into the dendrites, where it can play an important role in influencing whether the next incoming signal will set off the ion channels there.

Membrane properties limit the signal speed

Neurons are somewhat conductive, but not very conductive - they’re not copper wires. They’re also leaky - a large portion of their current seeps out into the surrounding intercellular fluid, partly because the cell membrane is not 100% impermeable to ions, and partly because of the membrane capacitance.

That means that if a given ion channel generates some voltage locally, that voltage will quickly drop off as we move farther from that channel.

So the ion channels cannot be spaced too far apart, and the signal will travel only as fast as the channels can open and close between the start and the end of the axon.

Myelin speeds up the signal

Vertebrates (including humans) have a clever innovation to speed up the propagation of signals: they use myelin sheaths to insulate axons that travel over distances longer than millimeters.

Myelin sheaths surrounding an axon of the peripheral nervous system (i.e. outside the brain or spine). Source: “Anatomy and Physiology” by the US National Cancer Institute’s Surveillance, Epidemiology and End Results (SEER) Program.

Myelin sheaths form in 0.2- 1 mm segments over the neurons and are made from layers of cell membrane and protein made by the dedicated cells - oligodendrocytes in the brain and spinal cord, and by Schwann cells everywhere else.

Myelin significantly thickens the axon’s wall and reduces the amount of current leaked out through the cell membrane. This allows the myelinated neuron to space their ion channels farther apart, significantly speeding up the propagation of the voltage spikes.

Newcomen engine is one of the first viable steam engines originally made in 1712 in England to pump water out of mines.

Unlike later steam engines, it was an “atmospheric engine” in which the power stroke was produced by the steam condensing into water and thus creating a vacuum; the atmospheric pressure then pushed the cylinder down, moving the big beam.

It was extremely inefficient by modern standards, but that wasn’t a big deal if it was sitting next to a coal mine.

Some fun details

• This engine is shown pumping the water out of a mine. The bulk of this water was discarded, but some was pumped up to a container at the top that would feed the engine.

• If too much air got into the main cylinder, it would stall. That’s why you see the engine leaking a bit of steam through a pipe when it’s inflating - to make sure that there’s minimum air in the main cylinder. The valve that the steam is leaking from was called the “snifting valve”.

• The big vertical plank that moves up and down is called the “plug tree”.

• The little stream of water on top of the cylinder is there to create an airtight seal.

• Fire is really painful to animate by hand if you don’t have any animation experience.

Not shown in the animation is an avanced feature of some of the engines: a pressure-controlled valve that helped refill the water in the boiler.

The animation above is rather stylized. This sketch from a 1868 book shows what it actually looked like on the inside:

Though typically the boiler, the pipes, and the switches would be enclosed in a brick structure like the one below. You can see the same engine in action in this YouTube video.