Joe's
Digital Garden

Software Engineering Lab Notebook

My notion of a Software Engineering Lab Notebook or just Lab Notebook comes from an article by Sam Bleckley1, but is also very similar to the work habits described by Kent Beck in Test Driven Development2 and the Knowledge Lakehouse described by Xinye Tao3.

The lab notebook serves several functions: 1) it gets everything out of our head so we can safely step away from a project, return later to it, review the last entries, and pick it back up; 2) it allows us to answer to clients what labor was done, when it was done, and how long it took; and 3) it provides a resource for documentation, that is, communication with team members on the state of a software system under work.

A lab notebook is not a retrospective. Entries are made before doing the work. Although, the occasional snapshot or summary entry make occur.

In it's simplest form, the lab notebook is an append only log (written in pen) of work over the course of a project that details:

  • What is the problem?
  • What what was tried
  • When it was done
  • What was discovered

What was discovered is a key item. We may discover that what was tried failed. Most often, we may discover that more work is needed and what that work is.

I begin each day my marking the change of day in the notebook. I read over the previous day's entries, move any unfinished work to today, and remark upon the problem I will begin solving, and what I will do to solve it. While working, I note any deviations to my plan and after I note any discoveries and note any corrections to past entries. Repeat.

One lab book per domain! Right now I have one for work, and one for personal projects and research. Each is a Leuchtturm 1917 bullet journal with the first few pages kept as an index and all subsequent pages after being the running log. Although, the workflow is also easily adapted to a simple text file.

Each spread in the log has a horizontal row across the top containing a title for the work conducted on these pages and the date when the spread was started. Each page has a four-square wide column on the right where dates and times are recorded. The remaining space is kept for logging entries.

It builds on the syntax of [[Bullet Journals]].

The left column denotes the date, and the time of the entry. A bullet indicates a task to be done.

Add Post Module                                                 2020-10-20 
---------------------------------------------------------------------------

10-20   |
--------|
0900    | • Add a post class

When that task is completed. I mark it completed with an x and then add another bullet for the next task. The key is to write down what you are going to implement before you embark on implementing it.

0900    | x Add a post class
0915    | • Write a unit test for post publication

As I am working a task, I might add notes to reference in the future using the - icon or if I discovered additional orthogonal work, I can add them using the + icon.

If old notes or additional tasks are rendered irrelevant, then I can strike through these tasks to remove them. If a note references an external source, I should note the title, author and year of publication for that source. Additional notes may also incorporate [[mind maps]] and UML diagrams.

0900    | x Add a post class
0915    | • Write a unit test for post publication
        | - Note posts are not organized temporarily
        | + Posts should have an author; add an author class

I can also use the circle ("◦") icon to represent scheduled events such as meetings:

0900    | x Add a post class
0915    | • Write a unit test for post publication
        | - Note posts are not organized temporarily
        | + Posts should have an author; add an author class
        | ◦ Meet with product to discuss author properties @1300

When I complete the current bullet that I am working on, I can either add another bullet for my next task or I can migrate one of my new tasks into a bullet:

0900    | x Add a post class
0915    | x Write a unit test for post publication
        | - Note posts are not organized temporarily
        | > Posts should have an author; add an author class
        | ◦ Meet with product to discuss author properties @1300
1000    | • Add an author class

I migrate a task by writing over the + with an > and then adding a new timestamp and bullet for the new task. This allows me to quickly look over the log and see all outstanding tasks since any + or • icon that has been neither completed, struck through or migrated is still outstanding.

If a blocker arises, I stop work on my current bullet and add a new timestamp and bullet for the blocker, then switch to working on the blocker:

0900    | x Add a post class
0915    | x Write a unit test for post publication
        | - Note posts are not organized temporarily
        | > Posts should have an author; add an author class
        | ◦ Meet with product to discuss author properties @1300
1000    | • Add an author class 
1015    | • /tmp is full on prod

When the blocker is finished. I migrate the original task to a new timestamp and bullet:

0900    | x Add a post class
0915    | x Write a unit test for post publication
        | - Note posts are not organized temporarily
        | > Posts should have an author; add an author class
        | ◦ Meet with product to discuss author properties @1300
1000    | > Add an author class 
1015    | x /tmp is full on prod
1020    | • Add an author class

It is conceivable for lots of tasks placed on hold to never be resumed, they become a backlog. If a backlogged item is ever resumed, we can run back through the log to when that task was placed on hold to see what work was done leading up to that hold. Mornings are a good time to review all new and on-hold items, migrate them to today, or strike them.

As you can see, the log allows me to quickly assess what work I was doing at any part of the day, what work remains outstanding.

External References

  1. Bleckley, Sam. Lab Notebooks <https://sambleckley.com/writing/lab-notebooks.html>. Retrieved 2023.
  2. Beck, Kent. Test Driven Development: By Example. O'Reily, 2002.
  3. Tao, Xinye. Introducing My Knowledge Lakehouse <https://tabokie.github.io/non-fiction/2024/01/13/log-kb.html>. Retrieved 2023.

Linked References