WebAssembly (WASM) is a binary instruction format designed as a portable compilation target for programming languages. Think of it as a compact, fast, sandboxed virtual machine that runs alongside JavaScript in the browser — and increasingly, outside it too.

The key properties that make WASM interesting:

• Fast: near-native execution speed, with predictable performance
• Safe: runs in a sandboxed memory space, isolated from the host
• Portable: the same .wasm binary runs on any platform with a WASM runtime
• Language-agnostic: you can compile Rust, C, C++, Go, and many other languages to WASM

graph LR
    A[Rust / C / Go] -->|compile| B[.wasm binary]
    B --> C[Browser]
    B --> D[Zed editor]
    B --> E[Node.js]
    B --> F[Cloudflare Workers]
    style B fill:#f9d71c,stroke:#333,color:#000

What makes WASM particularly compelling is the safety guarantee: since it runs in a sandbox, a host application can load and execute third-party WASM modules without worrying about them accessing the file system, network, or memory outside their allocated space. This is exactly why both browser extensions and code editors have adopted it as their extension format.

How I first heard about it

I first came across WebAssembly through Stu, a colleague, who shared a message about it on Slack:

On December 5, The World Wide Web Consortium (W3C) announced that the WebAssembly Core Specification is now an official web standard. This makes WebAssembly the fourth language for the web, following HTML, CSS, and JavaScript.

After sending this message on 26th December 2019, it planted a seed, but it felt like something reserved for game engines and image processing libraries — not the sort of thing I would use day to day.

What followed were three separate encounters with WASM, each teaching me something different. First, a machine learning model running entirely in the browser. Then, a Firefox extension where Rust makes decisions and TypeScript executes them. Finally, a Zed editor plugin where WASM is just a thin bootstrap layer. Together, they showed me that WASM is not one thing — it is a spectrum of integration patterns.

My first encounter: a small language model

My first real experience using WASM happened in an unexpected way. I was working on a project where we needed the browser to recognise images as part of a client’s Know Your Customer (KYC) process. We used a third-party service, and my job was to integrate our system with their API.

The third-party SDK shipped a machine learning model compiled to WASM. This model ran entirely in the browser — it could analyse an image of a passport or driving licence and extract the relevant fields, all without sending the image to a server. The WASM module handled the heavy computation (running the ML inference), while JavaScript orchestrated the camera feed and UI.

It was eye-opening to see that a machine learning model could run entirely on the user’s computer without communicating with a server.

This taught me something important about WASM: it’s not just a faster version of JavaScript. WASM allows you to use powerful tools and libraries from other languages, such as Rust, C, and Python, in places that previously supported only JavaScript. This led me to choose Rust and WASM when I needed more advanced features for my browser extension.

Stained Wall: a Firefox extension I built, powered by WASM

As you most likely know, I am Venezuelan and access to the news is often blocked by websites or not reachable from my country. Since I fled, I normally screenshot news articles and share them with my people who are still there so they can stay informed.

So, some of the news outlets I follow have gone to a paid subscription model, and I can’t access them from my country either. That’s why I built Stained Wall — a Firefox extension that helps me with that using WASM.

The architecture

It is a Firefox Manifest V3 (MV3) browser extension. Its architecture follows a clear principle: Rust decides, TypeScript executes.

The Rust/WASM engine handles all the logic — matching URLs against site configurations, generating bypass plans, and deciding which scripts to block. TypeScript handles everything that requires browser APIs — DOM manipulation, request interception, and messaging between the background service worker and content scripts.

flowchart TB
    subgraph "Background Service Worker"
        A[Page navigation event] --> B[WASM Engine.resolve url]
        B --> C{Plan found?}
        C -->|Yes| D[Send plan to content script]
        C -->|No| E[Set inactive icon]
        D --> F[Block scripts via webRequest]
        D --> G[Spoof User-Agent if needed]
    end

    subgraph "Content Script"
        H[Receive plan] --> I[Phase 1: Hide elements CSS]
        I --> J[Phase 2: Remove DOM elements]
        J --> K[Phase 3: Fetch archive / extract JSON-LD]
    end

    D --> H

    style B fill:#f9d71c,stroke:#333,color:#000

Why WASM for a browser extension?

You might ask: why not just write the matching logic in TypeScript? Three reasons:

1. Safety: Rust’s type system and borrow checker catch entire categories of bugs at compile time. The engine forbids unsafe code entirely (#![forbid(unsafe_code)]), so there are zero memory safety concerns in the WASM module.

2. Testability: The Rust engine can be tested natively with cargo test, without needing a browser environment. Unit tests for URL matching, plan generation, and script blocking run in milliseconds.

3. Separation of concerns: The engine knows nothing about browser APIs. It takes a URL string and returns a JSON plan. This makes it trivially easy to reason about: given this input, what output do I get?

Features

The extension supports a rich set of bypass strategies, all declared in a sites.json configuration file:

Each strategy is declarative — you configure it in JSON, and the Rust engine generates the appropriate plan:

{
  "name": "A fictitious site",
  "domains": ["something.com"],
  "strategy": {
    "archive": {
      "paywall_selector": "teg-page-wall",
      "article_selector": "main"
    },
    "hide": ["div[class*=\"adComponent\"]", ".wall-overlay"]
  }
}

The browser extension API

Working with the Firefox extension API in MV3 requires careful choreography. The WASM module must be loaded in the background service worker, but MV3’s Content Security Policy restricts how you can do this. The key pieces:

1. Declare WASM as a web-accessible resource (manifest.json):

{
  "web_accessible_resources": [
    {
      "resources": ["pkg/stained_wall_engine_bg.wasm"],
      "matches": ["<all_urls>"]
    }
  ]
}

1. Allow WASM evaluation in the CSP:

{
  "content_security_policy": {
    "extension_pages": "script-src 'self' 'wasm-unsafe-eval'"
  }
}

1. Load the WASM module using browser.runtime.getURL():

const wasmUrl = browser.runtime.getURL('pkg/stained_wall_engine_bg.wasm');
await init({ module_or_path: wasmUrl });
const sitesJson = await fetch(browser.runtime.getURL('sites.json'));
engine = new Engine(await sitesJson.text());

One subtle challenge is timing. The content script might load before the WASM engine has finished initialising in the background script. To handle this, Stained Wall uses a dual messaging pattern:

• Push: the background script sends the plan to the content script as soon as the engine resolves it
• Pull: the content script also requests the plan from the background script as a fallback

This ensures the plan reaches the content script regardless of initialisation order.

Safe DOM operations

MV3’s strict CSP means you cannot use element.innerHTML to inject content. All DOM operations in Stained Wall use DOMParser instead:

// Safe alternative using DOMParser
const parser = new DOMParser();
const doc = parser.parseFromString(html, 'text/html');
while (doc.body.firstChild) {
  element.appendChild(doc.body.firstChild);
}

This pattern is more verbose but eliminates any risk of XSS and keeps the extension compliant with MV3’s security model.

The TypeScript interface problem

One practical challenge I ran into while building Stained Wall: when you compile Rust to WASM with wasm-bindgen, you get auto-generated TypeScript bindings. But there is a catch: the boundary between Rust and JavaScript is limited to simple types. You cannot pass a rich Rust struct directly to JavaScript. Instead, you serialise to JSON on the Rust side and deserialise on the JavaScript side.

This means you end up maintaining parallel type definitions:

Rust side (plan.rs):

#[derive(Serialize)]
pub struct BypassPlan {
    pub site_name: String,
    pub hide_selectors: Vec<String>,
    pub block_script_patterns: Vec<String>,
    pub spoof_useragent: Option<String>,
    pub fetch_archive: Option<ArchivePlan>,
    // ...
}

TypeScript side (types.ts):

export interface BypassPlan {
  site_name: string;
  hide_selectors: string[];
  block_script_patterns: string[];
  spoof_useragent: string | null;
  fetch_archive: ArchivePlan | null;
  // ...
}

The JSON bridge is simple and debuggable — you can inspect the data flowing between Rust and TypeScript in the browser console — but keeping these types in sync is a manual process.

sequenceDiagram
    participant Rust as Rust (WASM)
    participant JSON as JSON bridge
    participant TS as TypeScript

    Rust->>JSON: serde_json::to_string(&plan)
    JSON->>TS: JSON.parse(planJson) as BypassPlan
    Note over JSON: Simple, debuggable,<br/>but types must stay in sync

Zed MJML: a code editor extension I built, powered by WASM

What is MJML?

MJML is an email markup language that compiles to responsive HTML email. If you have ever tried to write HTML emails by hand — with their nested tables, inline styles, and inconsistent client support — you will appreciate why MJML exists. It provides a component-based syntax that handles the responsive email nightmare for you:

<mjml>
  <mj-body>
    <mj-section>
      <mj-column>
        <mj-text>Hello world</mj-text>
        <mj-button href="https://example.com">Click me</mj-button>
      </mj-column>
    </mj-section>
  </mj-body>
</mjml>

I used to do this by hand in Ye Old days (2015).

Why I built a Zed extension

Zed is a modern code editor built in Rust, designed for speed and collaboration. Its extension system uses WASM as the plugin format — every extension is a compiled .wasm module that runs in a sandbox. I wanted proper MJML support in my editor, so I built zed-mjml — a language server extension that provides diagnostics, validation, and previews. This is where things get interesting and where the contrast with browser extensions becomes clear.

The Zed extension API

Zed extensions implement a Rust trait that gets compiled to WASM:

use zed_extension_api as zed;

struct MjmlExtension {
    cached_binary_path: Option<String>,
}

impl zed::Extension for MjmlExtension {
    fn new() -> Self {
        MjmlExtension { cached_binary_path: None }
    }

    fn language_server_command(
        &mut self,
        _language_server_id: &zed::LanguageServerId,
        worktree: &zed::Worktree,
    ) -> Result<zed::Command> {
        // Download and return path to mjml-lsp binary
    }
}

zed::register_extension!(MjmlExtension);

The API provides a set of platform functions that extensions can call:

• zed::current_platform() — detect OS and architecture
• zed::latest_github_release() — fetch release metadata from GitHub
• zed::download_file() — download files with gzip decompression
• zed::make_file_executable() — set the executable bit
• zed::set_language_server_installation_status() — show progress in the UI

The Zed pattern

Here is where WASM usage fundamentally differs from Stained Wall. In the browser extension, WASM runs the core logic. In the Zed extension, WASM is just a bootstrap layer. The extension’s sole job is to:

1. Check for the latest release of the mjml-lsp binary on GitHub.
2. Download the correct platform-specific binary (macOS arm64, macOS x86_64, Linux x86_64)
3. Make it executable
4. Tell Zed where to find it.

All the actual work — parsing MJML, validating documents, reporting diagnostics — happens in a native Rust binary that communicates with Zed over the Language Server Protocol (LSP).

flowchart LR
    subgraph "WASM sandbox"
        A[Extension loads] --> B[Check GitHub releases]
        B --> C[Download mjml-lsp binary]
        C --> D[Return binary path to Zed]
    end

    subgraph "Native process"
        E[mjml-lsp starts] --> F[Listen on stdio]
        F --> G[Receive document changes]
        G --> H[Validate MJML]
        H --> I[Send diagnostics back]
    end

    D --> E

    style A fill:#f9d71c,stroke:#333,color:#000
    style E fill:#4ecdc4,stroke:#333,color:#000

What is a language server?

A language server is a separate process that provides language-specific features to an editor: diagnostics (errors and warnings), autocompletion, go-to-definition, hover information, and more. The Language Server Protocol (LSP) standardises the communication between the editor and the server, so the same language server can work with VS Code, Zed, Neovim, or any editor that supports LSP.

The mjml-lsp, server communicates over stdio (standard input/output) using JSON-RPC messages:

sequenceDiagram
    participant Zed as Zed Editor
    participant LSP as mjml-lsp (native)

    Zed->>LSP: textDocument/didOpen
    LSP->>Zed: textDocument/publishDiagnostics
    Zed->>LSP: textDocument/didChange
    LSP->>Zed: textDocument/publishDiagnostics
    Zed->>LSP: textDocument/didClose
    LSP->>Zed: textDocument/publishDiagnostics (clear)

Tree-sitter: syntax without a custom grammar

One of the elegant decisions in the MJML extension is reusing the existing tree-sitter-html grammar. Since MJML is syntactically identical to HTML — just with custom tag names like <mj-section> and <mj-text> — the HTML parser handles it perfectly.

Tree-sitter is an incremental parsing library that generates a concrete syntax tree from source code. Zed uses it for syntax highlighting, bracket matching, indentation, and code folding. Instead of writing a custom MJML grammar, the extension uses query files (.scm files written in S-expressions) to teach Zed how to treat MJML-specific tags:

;; highlights.scm — Structural tags highlighted as keywords
((tag_name) @keyword
  (#match? @keyword "^(mjml|mj-head|mj-body|mj-include)$"))

;; Head configuration tags highlighted as types
((tag_name) @type
  (#match? @type "^(mj-attributes|mj-style|mj-font)$"))

;; CSS injection inside <mj-style> tags
(element
  (start_tag (tag_name) @_tag (#eq? @_tag "mj-style"))
  (text) @injection.content
  (#set! injection.language "css"))

This approach means the extension gets syntax highlighting, bracket matching, auto-indentation, and CSS injection inside <mj-style> blocks — all without writing a single line of grammar code.

The validation pipeline

The language server uses a two-pass validation strategy:

Pass 1 — Custom semantic validation: A lightweight byte-level scanner extracts tags and their relationships, then four rules check against the MJML specification:

1. Nesting validation: <mj-text> must be inside <mj-column> or <mj-hero>, not directly in <mj-section>
2. Required attributes: <mj-image> must have a src attribute
3. Unknown tag detection: Warns about typos like <mj-seciton> and suggests <mj-section> (using Levenshtein distance)
4. Singleton enforcement: Only one <mj-head> and one <mj-body> per document

Pass 2 — Structural validation: The mrml crate (a Rust MJML parser) performs deep structural validation, catching issues like unclosed tags and malformed XML.

flowchart TB
    A[Document changed] --> B[Scanner: extract tags]
    B --> C[Rule 1: Nesting]
    B --> D[Rule 2: Required attributes]
    B --> E[Rule 3: Unknown tags]
    B --> F[Rule 4: Singletons]
    A --> G[mrml parser]
    C --> H[Collect all diagnostics]
    D --> H
    E --> H
    F --> H
    G --> H
    H --> I[Send to editor]

Both passes run independently, and all diagnostics are reported together, giving the developer a complete picture of issues in their MJML document.

Differences between Zed and browser extension APIs

Having built WASM extensions for both platforms, the differences are striking:

The most significant difference is in what the WASM module actually does. In the browser extension, WASM is the brain — it makes decisions that drive the extension’s behaviour. In the Zed extension, WASM is the hand — it performs a mechanical task (downloading a binary) and then steps aside.

This difference comes down to platform constraints:

• Browser extensions have access to rich JavaScript APIs but need WASM for performance and safety. The WASM module can do substantial work because the browser provides a full JavaScript runtime alongside it.

• Zed extensions run in a stricter WASI sandbox with limited capabilities. The WASM module can call Zed’s API functions but cannot spawn processes, open network connections, or access the file system directly. For anything beyond simple logic, the pattern is to delegate to a native binary via LSP.

graph TB
    subgraph "Browser Extension Pattern"
        direction TB
        BW[WASM Engine] -->|JSON plan| BTS[TypeScript]
        BTS -->|browser.* APIs| BDOM[DOM / Network]
        style BW fill:#f9d71c,stroke:#333,color:#000
    end

    subgraph "Zed Extension Pattern"
        direction TB
        ZW[WASM Bootstrap] -->|binary path| ZLSP[Native LSP Binary]
        ZLSP -->|LSP protocol| ZED[Zed Editor]
        style ZW fill:#f9d71c,stroke:#333,color:#000
        style ZLSP fill:#4ecdc4,stroke:#333,color:#000
    end

What I learned

Building these two projects taught me that WASM is not a single thing — it is a spectrum of integration patterns. At one end, you have heavy WASM modules that perform substantial computation. At the other end, you have thin WASM wrappers that exist purely to satisfy a platform’s extension format (like the Zed MJML extension).

The key is understanding what the host platform offers and what it constrains:

• If the host gives you rich APIs and a JavaScript runtime, lean into WASM for the logic that benefits from Rust’s safety and performance.

• If the host gives you a strict sandbox with limited capabilities, use WASM as a bridge to native code where the real work happens.

WASM’s portability promise is real, but it is not about running the same code everywhere in the same way. It is about having a safe, sandboxed format that platforms can adopt as their extension mechanism — and then building the right architecture around it.

If you have not tried WASM yet, I would encourage you to start small. Pick a piece of logic that is well-defined and self-contained — a validator, a parser, a matcher — compile it to WASM, and see how it feels. You might be surprised at how natural it is to have Rust and TypeScript working side by side.

It is an open source project: zed-mjml.

Before diving into this blog, please check that the following prerequisites have been met.

1. Are you comfortable writing directly in WebAssembly Text? (Be honest)

   If the answer is “No”, then please read my Introduction to WebAssembly Text

2. Install wasmtime. This is an Open Source project by the Bytecode Alliance that provides both the WebAssembly development tools we will be using, and the WebAssembly System Interface (WASI) that will be the focus of our attention in this blog.

3. In order to understand how to code against the WASI interface, it is very helpful to look at the Rust source code that implements the WASI functions you will be calling from your WebAssembly Text program.

   This code can be found in the wasmtime GitHub repo https://github.com/bytecodealliance/wasmtime. The specific file to look in is crates/wasi-preview1-component-adapter/src/lib.rs

Explanation of Update

The original version of this program focused only on implementation the core the SHA256 algorithm. This was a good starting point, but it meant that the JavaScript wrapper used to invoke the WASM module had to read the file from disk, then write it to memory in the special format required by the SHA256 algorithm.

The purpose of this update therefore is to significantly reduce the degree of coupling between the JavaScript wrapper and the underlying WASM module by moving all the file IO into WebAssembly.

Overview of Steps

1. Getting Started
2. Start WASI
3. Import WASI Functions into WebAssembly
4. Count the Command Line Arguments
5. Extract the filename from the command line arguments
6. Open the file
7. Read the File Size
8. Do We Have Enough Memory?
9. Read the file into memory
10. Close the file

Extras

1. WebAssembly Coding Tips and Tricks
2. Debugging WASM

• But Why?
• Pulling Ourselves Up By Our Bootstraps
• Pulling Some Strings
• What Have We Achieved So Far?
• Extracting Characters From Keywords
• Tricks With Big Numbers
• So Where Are We Now?
• Tricks With Functions

Introduction

The functionality described in this blog is neither new nor is it unique. In this case, it is an extensive rewrite and optimisation of the original Hieroglyphy by Patricio Palladino.

Here is my version of Hieroglyphy.

Other variations of this style of app exist that use a minimal alphabet, but in this particular case, a close-to-minimal alphabet has been chosen.

WARNING
I can think of no practical reason why you would ever want to use this library in a real life situation…

🤪

But that said, the process by which it works is interesting if you really want to understand the inner workings of JavaScript’s type coercion behaviour

Overview

There has been some investigation into encoding the source code of a JavaScript program such that it uses a reduced alphabet, but remains syntactically valid and executable.

The object of the exercise here is not to create a program that remains human readable, but one that can be evaled and executed.

For example:

$ node
Welcome to Node.js v16.12.0.
Type ".help" for more information.
> eval('(+((+!![]+[])+(!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])))[(!![]+[])[+[]]+([]+{})[+!![]]+([]+([]+{})[([]+{})[!![]+!![]+!![]+!![]+!![]]+([]+{})[+!![]]+([][+[]]+[])[+!![]]+(![]+[])[!![]+!![]+!![]]+(!![]+[])[+[]]+(!![]+[])[+!![]]+([][+[]]+[])[+[]]+([]+{})[!![]+!![]+!![]+!![]+!![]]+(!![]+[])[+[]]+([]+{})[+!![]]+(!![]+[])[+!![]]])[!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]]+(!![]+[])[+[]]+(!![]+[])[+!![]]+([][+[]]+[])[!![]+!![]+!![]+!![]+!![]]+([][+[]]+[])[+!![]]+([]+([]+{})[([]+{})[!![]+!![]+!![]+!![]+!![]]+([]+{})[+!![]]+([][+[]]+[])[+!![]]+(![]+[])[!![]+!![]+!![]]+(!![]+[])[+[]]+(!![]+[])[+!![]]+([][+[]]+[])[+[]]+([]+{})[!![]+!![]+!![]+!![]+!![]]+(!![]+[])[+[]]+([]+{})[+!![]]+(!![]+[])[+!![]]])[+((+!![]+[])+(!![]+!![]+!![]+!![]+[]))]](+((!![]+!![]+!![]+[])+(!![]+!![]+!![]+!![]+!![]+!![]+[])))+(!![]+[])[!![]+!![]+!![]]+(![]+[])[!![]+!![]]+(![]+[])[!![]+!![]]+([]+{})[+!![]]')
'hello'
>

Room from improvement

This library is just a proof of concept; there is plenty of room for improvement:

• Ensure that all characters have been encoded using the shortest possible representation
• Allow some flexibility in the alphabet size to account for different target runtime environments
• Knowing the specific runtime would allow us to take advantage of features unique to that environment, which in turn, may yield shorter character encodings

• SHA256 Algorithm Overview
• WebAssembly Does Not Have A “raw binary” Data Type
• WebAssembly Program Architecture
• WebAssembly Implementation
• Unit Testing WebAssembly Functions
• JavaScript Host Environment
• Summary

Development Objectives

1. See how small a binary can be produced when the SHA256 digest algorithm is implemented directly in WebAssembly Text
2. Compare the runtime performance of the WebAssembly module with the native sha256sum program supplied with macOS

The Git repo containing the working software can be found here

Development Challenges

Two challenges had to be overcome during development:

1. The SHA256 algorithm expects to handle data in network byte order, but WebAssembly only has numeric data types that automatically rearrange a value’s byte order according to the CPU’s endianness.
2. Unit testing WASM functions within a module is an entirely manual process. This presented an interesting challenge - especially when writing unit tests for private WASM functions

Tim Lee — 16 December 2022

Introduction

Short notes from a 2-day Principal Dev training I attended, lead by Eduards Sizovs.

Full course slides can be found here (arrow to navigate).

Short notes

Agile

• Agile manifesto says “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software”. But customers don’t need software, they need cost-efficient solutions to their problems. So perhaps it should be reworded to: “Our highest priority is to find cost-efficient solutions to customers’ problems”
• Process of finding the cost-efficient solution from a list of options is called Cost-Benefit Analysis (CBA). Parameters to consider - speed, cost, benefit, risk
• Disney brainstorming method good for sourcing ideas as a group
  1. Phase 1: gather all ideas, acting as “Dreamers”
  2. Phase 2: assess ideas as “Realists” looking for limitations
  3. Phase 3: analyze ideas as “Critics”, addressing possible risks
• Don’t do what a customer says they want, do what a customer needs (cost-efficiently). Should solve the problem, not the want.
• Important not to shield developers from the business, because software is the implementation of the business. Shielding leads to low trust, low motivation and mediocre software solutions.
  • To find cost-efficient solutions, you need a whole-team approach.
  • To write great code, you need a deep understanding of the underlying problem. (software is the implementation of the problem domain. Weak problem understanding -> weak solutions)
  • Understanding the business domain turns engineers into business partners
  • The team owns the product, not product owners. Product ~owners~ leaders promote product understanding.
  • Make impacts, not software

Leading

• As a leader your job depends on seeing things others don’t see. Can’t make yourself always busy - need head space to observe.
• Should switch focus to high-leverage activities (HLAs) - the activities with the highest return on your time investment. Leverage = impact / effort
• Your output is the output of your team. Being busy is not the same as being productive. So focus on HLAs (Pareto principle - 20% of activities give 80% of result). To notice HLAs, don’t overwhelm yourself with Low Level Activities (LLAs).
• To get rid of LLAs: Delete -> Defer -> Delegate -> Do
• Removing impediments is good, but empowering the team to see, prioritise and remove it’s own impediments is better.

Measuring dev team’s performance

• Lead time - time taken to solve a problem, or time a problem stays “in progress”
  • Can be reduced with cost efficient solutions, good architecture, full-cycle teams (autonomous, cross-functional, self-organising)
• Defects
• DDP (due date performance) - ability to deliver on time
• Reliability
  • MTBF (mean time between failures), MTTD (mean time to detect), MTTR (mean time to remediate)
• ROI - what can you offer that the market can’t? Cost-efficient solutions, cost reduction, innovation, leadership, mentoring, visibility, hiring
• Satisfaction - of customers and business
  • Ask 3 questions:
    1. Are you happy with my work?
    2. What do I suck at?
    3. What can I do to improve?

Throughput

• Little’s law - lead time = work in progress / throughput
• Reducing WIP leads to work being delivered more quickly
• Focus as many people as possible on as few projects/stories/tasks as possible - swarming.
• 4 ways to increase throughput:
  1. Hire more people - though law of diminishing returns (adding more people beyond certain threshold is less efficient). Also Brooks law - adding manpower to late project makes it even later. Small teams tend to do better than large ones - more flexible, aligned, involved, engaged, better relationships, low management overhead. If need more people, continuously refactor company into a network of small, independent, full-cycle teams
  2. High-performance teams - aligned, small, diverse, full-cycle, stable (to achieve Tuckman’s stages of group development - forming, storming, norming, performing, takes about a year to get to last stage), skilled
  3. Technical excellence - engineers learn by example from seniors, so choose seniors and leaders carefully. Make sure they promote software craftmanship, clean code, extreme programming, TDD. Make mentoring a prerequisite for promotion.
  4. Reduce waste, which includes
     1. Partially done work
     2. Overproduction
     3. Waiting/handoffs
     4. Rework
     5. Non-value added activities

Tips for delivering on promises (namely, within Scrum)

• Know your velocity (work delivered without cutting corners)
• Even under pressure, don’t take more work than allowed by your velocity
• Remove planned work when unplanned work appears
• Minimise unplanned work
• Estimate with Fibonnaci numbers to gain speed at cost of accuracy
• Play planning poker until all team members estimates match to reduce HIPPO pressure
• When estimation is difficult or the problem is too big - split or spike to reduce unknowns
• Estimates are innacruate - comfortable velocity must include a buffer
• Turn on swarming for speed and continuous flow, monitor flow to notice issues
• Keep tech debt under control
• Process of estimation is more important than estimates - learn about business, understand problems better, cut scope, slice work for better flow, raise inconvenient questions like why estimates are inflated…

Tech debt

• There is a strong correlation between the quality of the codebase and throughput.
• A delivery team tries to maintain it’s velocity over time.
• Degredation of the codebase accelerates with time due to reinforcing loops - broken window theory, Brook’s law, Lehman’s law (entropy), Gresham’s law (bad code drives out good - you want to touch nice code, you don’t want to touch bad code so they stay longer and sometimes have abstractions written around them).
• Way to achieve sustainable pace/continuous improvement - every time you touch code try and make it slightly better, never worse than before (the Boyscout rule)
• Currency of technical debt is throughput - you borrow throughput and pay it back. It compounds, so the longer it takes to pay back, the worse it gets. Things get built on top of it. So want to pay technical debt early.
• Tech debt is a tool for short term gain.
• Explain the cost of tech debt when making the trade off so there’s an understanding that it will either slow down future delivery or be paid back soon.
• Eliminating the cause of technical debt is more important than eliminating the technical debt.
• 4 main causes of tech debt:
  1. Knowledge or skills problem - strong technical leadership/mentoring needed
  2. Attitude problem - “I don’t have time for quality”
  3. Borrowing
  4. Learning - knowing more at a later date, finding a better way to do it. So important to attract, develop, retain tech/domain knowledge, along with KISS, YAGNI, validate before building

Mentoring

• The learning pyramid goes from least effective to most effective - watching lectures -> reading -> audio/visual -> demonstration -> group discussion -> practicing by doing -> teaching others
• Pyramid moves from passive learning method to active learning methods
• Teaching others is most effective, notably even more than by doing, which is why mentoring is crucial
• 4 stages of learning:
  1. Accumulate knowledge (read, hear, practice)
  2. Teach, explain (code review, pairing, writing, presenting)
  3. Get feedback
  4. Deepen understanding (and improve how well you can articulate an idea)
• If you can’t explain it, you don’t understand it

High performing teams

• Bus factor - make yourself replaceable. The more influential your role, the more you have to care about succession planning. Leaders grow leaders.
• Team performance = sum of performance of each individual x teamwork / wastes
• The wider the skill gap between you and your teammates, the more you have to mentor
• A team needs T-shape people, ensure people have some capability in all areas
• Retention more important than hiring - retain domain knowledge, benefit from growing them, don’t have cost of hiring
• 2 types of motivation
  1. Intrinsic - driven because you find it rewarding, performing an activity for its own sake, the behaviour itself is its own reward
  2. Extrinsic motivation - driven to earn a reward, get praise or avoid punishment. Do something because expect to get something in exchange
• Intrinsic motivation is the optimum, more sustainable driver.
• Primary motivators:
  1. Autonomy - freedom to make decisions, create workspace, choose what to work on. Not complete freedom, especially more junior members - not earned trust. More senior = more autonomy - more trust, more expertise.
  2. Mastery - becoming more capable, learning new skills
  3. Purpose - aligning work with personal goals

Recruiting

• Important to improve quality of inbound traffic
• Be visible, otherwise people will come to the interview just to find out who you are, waste of time
• Write ads that attract good candidates (with right attitude but lacking certain skills, or with imposter syndrome) and drives away bad candidates (right skills but lacking attitude)
• Perceived ability vs Actual ability graph. Imposter syndrome - high actual ability, low perceived ability. Dunning-Kruger - high perceived ability, low actual ability. A lot of job ads attracts Dunning-Kruger’s. Instead of “deep understanding”, “good knowledge of”, “+3 years of experience”, describe the environment, technology and what the person will be doing, what you tend to favour (pair programming, etc) and that it doesn’t matter if you don’t know something because they offer mentoring. Show job ads to as many colleageus as possible, not just the hiring manager/recruiter having written. textio.com - accessibility language checker
• Beat expectations
  • People should walk out of a conversation smarter, happier, energized (even if they don’t make the hire)
  • Invite for a conversation, not an . Run as a conversation instead of a conversation.
  • Start with a presentation
  • Inspire with gift of books in the interview that are relevant - e.g. Clean Code
• Let candidates code in their comfortable environment
  • The Hawthorne effect (Thinking, fast and slow) - when in stressful situation System 1 dominates and can’t think
  • Homework is most inclusive way to assess tech skills
  • Compact but representative
  • Owe the candidate a code review (good parts, bad parts, suggestions, books)

Reciprocity

• Reciprocity is a social norm of responding to a positive action with another positive action, rewarding kind actions. If you invest in someone, they’ll invest in you - invest wisely
• Important for:
  1. Authority - brings influence
  2. Consistency - basic social contract, do what you say
  3. Liking - if people don’t like you, they’ll object to your ideas even if they’re rational
  4. Scarcity - things that are available for everyone have lower perceived value. Things that are not available for everyone have higher perceived value. If you have skills that are more in demand, they’ll be more valuable.
  5. Social proof - shortcut to make decisions based on things like ratings, reviews, recommendations, etc

Management toolbox

• Encourage - help people fight self-doubts
  • #1 reason people leave their role is lack of appreciation - just say thanks is not good, say thank you, “name” for doing “x”
  • Support initiative
  • Behind every request is an unfulfilled desire or need - ask why if don’t understand, try to get to bottom of need
• Challenge - delegate technical and non-technical work
  • Delegate tasks that maximize learning (trade-off with speed)
• Answer “how can I do X or Y” with “what options do you see?”
• Share:
  • Knowledge
  • Energy - should know the source of your energy, understand what energises you
• Set rules
  • Less control, more systems
  • e.g. WIP limits, YBIYRI (You Build It, You Run It), CI with daily push into the , QA should find nothing (build quality in), every time new pair
• What skills do my team-mates need the most?
  • At the intersection of personal career goals and team needs
• Trust - product of:
  • Your knowledge
  • Your consistency
  • Your authority
  • Your confidence
  • Your charisma - energy, body language, voice, words
  • Read: Nonviolent communication
  • Your Emotional Intelligence (EQ) - managing thoughts, emotions, mind, ability to connect with others

Answering the question - am I a good leader?

• Is your team successful?
• Is each individual in your team successful?
• Is your work appreciated by others?
• Is your team reporting high scores in weekly health checks (Spotify strategy - R/G/B scored against categories)?
• Is employee turnover low?
• Are people lining up to work with you?

• A WebAssembly module and its host environment can share a block of linear memory.
• If JavaScript acts as the host environment, then shared memory appears as a JavaScript ArrayBuffer.
• JavaScript cannot directly manipulate the contents of an ArrayBuffer. Instead, it must use some sort of overlay or mask such as a Uint8Array or a Uint32Array. Then the data in the ArrayBuffer can be accessed using the overlaid structure’s semantics.

Problem Summary

A collision between these two facts creates the “Detached ArrayBuffer” problem:

• JavaScript ArrayBuffers are of fixed-length and once allocated, cannot be extended.
• WebAssembly linear memory can be extended by calling memory.grow.

If WebAssembly memory grows,¹ then the old JavaScript ArrayBuffer must be thrown away and a new one created. Consequently, any JavaScript objects that used to overlay the old ArrayBuffer are immediately invalidated because they now point to nothing. The floor has literally been pulled out from underneath these objects and they must all be redefined over top of the new ArrayBuffer.

There is a proposal to allow a JavaScript ArrayBuffer to grow, and as soon as this functionality appears, this problem will disappear.

Meanwhile, back in Gotham City…

What Consequences Do These Facts Create When Writing In Rust?

When writing a Rust program that you intend to distribute as a WebAssembly module, cargo knows that memory growth might be required; therefore, it builds the necessary functions into the WebAssembly module for calling memory.grow. Should it be necessary, memory growth will now happen automatically (and silently!)

The consequences for JavaScript are that its shared memory ArrayBuffer now points to a completely new block of memory and all the overlay objects that gave access to the “pre-growth” shared memory are no longer usable (I.E. they are said to have become “detached”).

If you then attempt to access shared memory using one of these “pre-growth” objects, you will see an error such as this:

TypeError: Cannot perform %TypedArray%.prototype.slice on a detached ArrayBuffer

Aside

Before you can compile a Rust program to WebAssembly, you must first install the wasm32 compilation target:

rustup target add wasm32-unknown-unknown

Local Execution

The following trivial application demonstrates this problem.

A WebAssembly program shares a block of memory with its host for the pourposes of data exchange. The host writes data to known locations in memory, then the WebAssembly program processes it and writes its response back at another known location.

Source Code

All the source code referenced by this blog can be found in the Github repository detached_arraybuffer.

If you wish to run these tests locally, first clone this repo:

git clone git@github.com:ChrisWhealy/detached_arraybuffer.git

First, Generate the WebAssembly Module

Testing can be performed using different versions of the Wasm module. One version will work because it does not perform memory growth, and the other will break because it does:

1. Compile a working version from source code written in WebAssembly Text.

   This version works simply because the WebAssembly Text source code was hand-written, and no such calls to memory.grow were implemented. To use this version, run

   wat2wasm memoryguest.wat
   

2. Compile a broken version from source code written in Rust

   To use this version:

   • Rename ./src/lib_growth.rs to ./src/lib.rs
   • Run cargo build --target=wasm32-unknown-unknown

3. Compile a working version from source code written in Rust that explicitly avoids the need for memory growth

   To use this version:

   • Rename ./src/lib_no_growth.rs to ./src/lib.rs
   • Run cargo build --target=wasm32-unknown-unknown

Test The Wasm Module By Calling It From JavaScript

The effects of WebAssembly memory growth on JavaScript’s shared memory ArrayBuffer can be demonstrated as follows:

1. In both server.js and client.js, ensure that the variable wasmFilePath points to the particular Wasm module you wish to test.

2. To test the Wasm module server side, run

   node server.js
   

3. To test the Wasm module in a browser:

   • Start a temporary Web Server

      python3 -m http.server 8080
     

   • Point your browser to http://localhost:8080
   • Open the developer console

When the test succeeds, the console will display

Ahoy there, Testy McTestface!

When the test fails, the console will show the Type Error shown above.

Implementation

The map of shared memory looks like this:

The JavaScript program must first obtain the values of the memory locations shown above. Once it has these, it writes the appropriate strings to those locations.

Next, it calls the Wasm function set_name which does the following:

• Combines the salutation and name into a greeting
• Writes that greeting to another known memory location
• Returns the length of the formatted greeting

Finally, the JavaScript program reads the greeting from shared memory and writes it to the console.

But What Caused Memory Growth?

Look at the Rust coding in ./src/lib_growth.rs. Within function set_name, the format!() macro is used to assemble the result, which is then stored in an intermediate String called greeting.

#[no_mangle]
pub unsafe extern "C" fn set_name(sal_len: i32, name_len: i32) -> i32 {
    let sal: &str = str_from_buffer(SALUT_OFFSET, sal_len as usize);
    let name: &str = str_from_buffer(NAME_OFFSET, name_len as usize);

    let greeting: String = format!("{}, {}!", sal, name);
// snip...

Well that looks harmless enough…

However, the declaration of the new String requires more memory than is currently available; so, using the extra functions generated by cargo, shared memory is automatically and silently extended.

As far as Rust (WebAssembly) is concerned, everything is fine; however, the JavaScript host environment sees that shared memory has changed size, so it throws away the old ArrayBuffer and helpfully creates you a new one.

And now all your “pre-growth” JavaScript references into WebAssembly’s shared memory are broken…

Calling The Broken Code From JavaScript

Look at ./server.js to see the full context of this coding.

const salutation = "Ahoy there"
const name = "Testy McTestface"

// Treat shared memory as an array of unsigned bytes
const mem8 = new Uint8Array(wasmExports.memory.buffer)

// Fetch long-lived pointers
const sal_ptr = wasmExports.get_salutation_ptr()
const name_ptr = wasmExports.get_name_ptr()
const msg_ptr = wasmExports.get_msg_ptr()

// Store salutation and name at the expected locations
mem8.set(stringToAsciiArray(salutation), sal_ptr)
mem8.set(stringToAsciiArray(name), name_ptr)

// Tell Wasm to write the formatted greeting to the known memory location then return its length
let msg_len = wasmExports.set_name(salutation.length, name.length)

// Read greeting from shared memory
let msg_text = asciiArrayToString(mem8.slice(msg_ptr, msg_ptr + msg_len))
//                                ^^^^^^^^^^ mem8 will point to nothing if memory growth occurs!

console.log(msg_text)

So let’s run this.

If you’re using the working WebAssembly module, you’ll see:

$ node server.js
Ahoy there, Testy McTestface!

and if you’re using the WebAssembly module that breaks JavaScript’s shared memory references, you’ll see:

$ node server.js
/Users/chris/Developer/WebAssembly/detached_arraybuffer/server.js:60
    let msg_text = asciiArrayToString(mem8.slice(msg_ptr, msg_ptr + msg_len))
                                           ^

TypeError: Cannot perform %TypedArray%.prototype.slice on a detached ArrayBuffer
    at Uint8Array.slice (<anonymous>)
    at /Users/chris/Developer/WebAssembly/detached_arraybuffer/server.js:60:44

Two Solutions

Until JavaScript’s ArrayBuffer is able to perform in-place growth, we must adopt one of two possible approaches to solving this problem. Either:

1. We monitor the size of the WebAssembly memory looking for growth; or
2. We adjust the Rust coding so that memory growth does not occur.

1. A JavaScript Workaround

If it’s going to change, WebAssembly memory will only every increase in size. So a simple way to workaround this problem is to monitor the size of the WebAssembly’s memory.

If it gets bigger, then you know you need to redefine any shared memory overlay objects.

This is just a workaround; it does not change the underlying problem.

Anyone else calling the same WebAssembly function will need to implement the same workaround.

The code does not require much modification to avoid using a possibly detached ArrayBuffer:

const salutation = "Ahoy there"
const name = "Testy McTestface"

// Keep track of Wasm's shared memory size
let memLength = wasmExports.memory.buffer.byteLength

// Snip

// Tell Wasm to write the formatted greeting to the known memory location then return its length
let msg_len = wasmExports.set_name(salutation.length, name.length)

// Before allowing shared memory access, check if memory growth has occurred
if (wasmExports.memory.buffer.byteLength > memLength) {
  memLength = wasmExports.memory.buffer.byteLength
  mem8 = new Uint8Array(wasmExports.memory.buffer)
}

// Read greeting from shared memory
let msg_text = asciiArrayToString(mem8.slice(msg_ptr, msg_ptr + msg_len))

console.log(msg_text)

Now everything works because we’re on the lookout for memory growth and then “reattach” the mem8 array to the new shared memory ArrayBuffer.

2. Solve the Problem in Rust

However, to avoid causing inadvertent memory growth, the Rust coding needs to avoid invoking any instructions that might require extra memory. In this case, it means that instead of using an intermediate String object, we write the bytes of the character strings directly to the [u8] buffer.

The full solution can be seen in ./src/lib_no_growth.rs, but the important change is shown below:

pub unsafe extern "C" fn set_name(sal_len: i32, name_len: i32) -> i32 {
    let mut idx: usize;

    // Write salutation directly to the buffer
    copy_bytes(MSG_OFFSET, SALUT_OFFSET, sal_len);
    idx = MSG_OFFSET + sal_len as usize;

    // Write separator ", "
    BUFFER[idx] = COMMA;
    idx += 1;
    BUFFER[idx] = SPACE;
    idx += 1;

    // Write name
    copy_bytes(idx, NAME_OFFSET, name_len);
    idx += name_len as usize;

    // Write bang character
    BUFFER[idx] = BANG;
    idx += 1;

    (idx - MSG_OFFSET) as i32
}

In all but the smallest of teams code reviews are a commonplace practice. The primary purpose of them is to ensure that the code is of good quality and meets the team’s technical standards of fitness.

Some of the things a reviewer could look for could include:

• Proper grammar and syntax
• Efficient algorithms
• No duplicate code
• Adherence to naming conventions
• Test coverage

… and so forth.

I think everything I’ve described above is pretty uncontroversial yet In spite of all of this I think code reviews today are (with some justification) a somewhat maligned practice.

Code reviews and their efficacy rely heavily on the existing relationships between the team members. Unhappiness with technical direction and personal animosities can spill over into the review process and end up harming team performance.

Conversely, teams that get along and have a solid and agreed vision of what good code looks like never seem to suffer these pitfalls and the process is a routine and effective guarantor of code quality.

To explore some of the complexities around code reviews we first need to examine its origins and recall how software used to be shipped.

How things were

In the past releasing new software was a chore. Preparing the release candidate, notifying internal stakeholders and users about it, weeks of manual regression testing and warnings of downtime on the day of release were all part and parcel of the process. Fear of what could go wrong loomed large over the whole process.

Since the cost of change was so high, getting things right the first time was of cardinal importance. A botched deployment meant kicking off the same lengthy process all over again to revert the failed change.

Developers were constrained by the tools and methodologies of the time - being totally confident in the success of a deployment was near impossible. Test driven development, a now widely accepted practice for writing resilient code wasn’t developed till the late 90’s by Kent Beck yet code was already written for devices large and small long before this.

In these times code reviews were a vital tool to ensure code quality. Seasoned engineers would pour over the proposed changes looking for syntax errors, potential compilation issues or any other gaps in code quality that could derail a release and result in a bunch of upset customers or users. It was manual, laborious and the stakes were high if something slipped through.

How things are now

The tools and methodologies used to ensure quality code have gone from sparse to abundant.

Developers have their choice of rich IDE’s that offer access to a vast plugin ecosystem for most any active programming language, providing convenience features like syntax highlighting and code formatting and more advanced features like code completion.

Testing frameworks are faster and more feature rich. With a solid testing strategy, developers can have a high degree of confidence that their new code has the intended effect without new bugs or regressions cropping up.

Today’s devs can even provision production-like services on their development machines through techniques like containerisation, providing lean standalone services - the kind of thing that would have been unimaginable for complicated subsystem teams of the past.

For companies that have leaned into continuous delivery and continuous integration as practices, code deployments that once took hours or days could be achieved in mere minutes. Tools like Github Actions can run automated checks against our bundled software before deploying it to production. Some of these checks can include automated tests (unit, e2e, contract etc..), static analysis tools, vulnerability scans just to name a few.

So what does this all mean?

The code review has gone from being one of a handful of practices to ensure code quality to just one amongst many. Verifying that our code can be safely merged is now also possible through another rival practice - pair programming.

Shifting left

The traditional way in engineering teams would solve coding problems was pretty straight forward - analyse the problem, write code to fix it then test the code. This is an approach which fits neatly into the roles of business analyst, developer and QA. Actual testing only came at the end.

The problem with this approach was that oftentimes requirements were poorly understood or missed entirely and too often devs would end up building the wrong thing. This left QA’s in an unenviable position of either sending time sensitive work back to developers to fix, potentially risking a deliverable, or letting it slide and creating a separate bug to track whatever got missed.

In 2001 Larry Smith proposed a different approach. He coined the term “shift left testing” to describe testing in smaller iterations. Instead of writing code and only testing it once it was done, he advocated for testing during development, in smaller iterations, so together the dev and QA could both be more confident that what’s being built would match what was asked for.

Around the same time, the Extreme Programming (XP) philosophy was gaining popularity. It challenged teams to take ownership of the software they ship, and to not view their role in the team as being siloed just to their job title. Far from just being on the shoulders of QAs and testers, ensuring the quality of the final product was becoming a whole team concern.

Baking in quality from the start would prove to be a powerful idiom and expanded to other areas of software delivery, particularly around non functional requirements. Testing, accessibility, observability, security - all of these that were traditionally regarded as something that was considered “once development was done” but are now seen as critical requirements that have to be considered up front.

Pair programming was conceived in the same spirit.

Having two developers working at the same terminal has existed for quite a while but true pairing - where the design, problem analysis and writing of code is conducted by peers that discuss approaches and alternate roles - would take time to grow in acceptance.

While controversial to some, pairing represents a shift left in the approval process.

Instead of having discrete write-code, review-code blocks - the two phases are merged into a single process where code is co-authored by two devs. This satisfies the “two pairs of eyes” principle and so any code they write could go to production - which is why it’s often used in conjunction with trunk based development.

In spite of the difficulty introducing pairing and trunk based development to engineering teams that haven’t used them before the benefits can be huge - with automated testing and CI/CD pipelines, teams can ship quality code many times a day with high confidence and none of the waiting around and context switching that characterises the humble code review.

What’s wrong with code reviews?

Before exploring how we can do code reviews better we need some appreciation of what are the practical drawbacks using code reviews as a means of safeguarding code quality in the first place.

I’ll group my critique of code reviews into two broad categories:

• The process itself
• Developer attitudes towards them

Code reviews as a process

Utilising code reviews means developer workflows are habitually broken.

If you’re writing code then you need to put out a request for a review and await the outcome of it before resuming work. If you’re the reviewer then your work is potentially being interrupted by being asked to conduct a review. These shifting and exchanging responsibilities are a form of context switching - dropping what you were doing before and focusing on something else.

Proponents of code reviews would argue the original submitter could just do something else while the review is ongoing but this too isn’t ideal. Having devs split their attention between multiple streams of work is inefficient and risks the quality of one or both being compromised.

Code reviews also contain within them an innate antagonism, forcing teams to choose between one undesirable consequence and another - and that is what is the ideal size of a pull request?

Small commits are generally considered to be good practice in coding. They encourage developers to be deliberate in their choices and result in more focused changes. Having pull requests with fewer commits mean they reveal their intention more clearly too, which makes reviewing them far more straightforward.

However, pull requests consisting of only one or two commits don’t gel very well with the manual nature of code reviews. If I produce multiple pull requests in a day then I’ll require a lot of reviews, which means lots of context switching for those on my team who have to review my work (and maybe some desensitisation towards my repeated entreaties for a review too)

But what about batching together many commits so instead of many small ones you just have one big one? You can do this but it has a cost attached.

Blending many commits into a single pull request means that the surface area of change is much greater, leading to a proportionally higher review time. While good test names and commit messages can mitigate this, the overall goal of what’s being achieved can become blurred, mandating a slower line-by-line approach to reviewing code to see what’s changed and where. Overly stuffed pull requests do not clearly reveal their intention like their smaller counterparts.

Or even worse if the surface area of change is massive then something could just slip by entirely e.g. renaming a file while also making business logic changes that could be lost in the revision comparison tool. Being “too big to review” is a risk too.

Ultimately teams need to come to their own conclusions about how big or small code reviews should be but either way there’s a cost involved regardless.

Developer attitudes to code reviews

Prevailing attitudes within engineering teams towards the review process itself can help or hinder the process as a whole. We understand that code reviews are primarily a technical assessment to assess the fitness of some code though I would argue it has a psychosocial facet to it too.

What does good code look like? Who is in a position to judge good from bad? What technologies or technical practices are desirable or not?

These are all subjective questions that don’t have hard answers. In the absence of some socialised view of what “good” looks like, developers will bring their experience to bear on the review - they also bring their biases, subjectivities and sometimes their egos too.

Commercial software development is a time consuming and expensive process. Meeting milestones on product roadmaps can take a long time to hit and forecasting can be like crystal ball gazing at best of times. For businesses continuous feature delivery is incredibly important, if not essential to their survival.

However this urgency is not always reflected in the spirit with which code is reviewed. Spurious or ineffectual review comments go beyond simply being annoying and if unchecked can be harmful and erode team trust.

Is there a better way?

Despite all these drawbacks I do think code reviews can be made to work.

However I think doing them effectively means engineering teams re-evaluating governance of the review process. Useful new tools can also help as previous tedious manual checks and be neatly automated away.

We also need to evaluate the human impact of the review process so we’ll touch on how giving effective feedback can guide us to a better, more empathetic review process.

Team owned quality

Some of the drawbacks of code reviews can seem quite apparent but others are much more subtle. One such subtlety is who owns code quality? The natural answer seems to be the senior most developers and technical leaders with the engineering teams. Their experience makes them natural arbiters of code quality and so it’s natural to assume that they should play an oversized role in making sure the code is up to scratch.

While this all seems quite reasonable I think this is a backward and self-defeating approach.

It’s somewhat uncommon for engineering teams to be staffed entirely with experienced developers. The commercial demands placed on organisations mean that there is always more work to be done than developers to do the work.

For this reason we see the emergence of leveraged teams - that is to say teams with one or two highly experienced developers, some with moderate experience and then some more junior developers to round out the team.

I would propose that having code quality owned by a handful of experienced developers is harmful to both the experienced devs and the mid/junior developers they are seeing to grow and teach:

Tech leads and principal developers often have responsibilities that lie outside the immediate concerns of the delivery team. Some include: delivery assurance discussions, product discovery, line management and cultivation concerns, tech analysis sessions, engineering working groups, tech visioning discussions and so forth. In short, they have a lot on their plate outside of coding (if indeed they have time to code at all).

On top of all this, if they alone are required to approve the code their team generates then it’s easy for them to become a single point of failure. Code reviews take time to effectively review and delay is expensive. Equally as someone that isn’t writing much code they might actually have less context on the codebase should be doing than other engineers that do nothing but write code all day.

This format is harmful for less experienced developers too. Being unable to influence coding standards risks them becoming disengaged and seeing quality as something that is the responsibility of QA’s and senior developers, not themselves. Why care about standards you have no agency to change? This is also harmful to their growth and could prompt talented developers to actively look for opportunities somewhere else where they can have more autonomy and influence.

The solution to these problems is team owned quality.

Engineering teams should get together early after forming and talk about coding standards in a collaborative and inclusive way. This should continue for the life of the team. The meeting should be a discussion where everyone has a chance to contribute and debate what makes for good code.

Senior developers should help guide the conversation but not act as gatekeepers of what is good and what isn’t and should approach discussions with an open mind. Though junior developers are new to their career, they can produce useful insights and fresh perspectives.

These sessions should be repeated weekly or fortnightly, with discussion points brought in eg. “can we adopt this new technology?”, “does this approach seem off to anyone else”, “Is it just me or are our pipeline tests flaky?”. As with all well run meetings, notes should be taken and actions assigned to individuals to follow up on before the next meeting where they recap their progress.

The offshoot of this is that everyone has an equal responsibility to ensure the quality of the software the team writes. If indeed a bug is released, it can be included as a discussion point in the next meeting (in a candid but blame-free way) and the team can come up with actions on how to prevent a similar event from happening in the future.

This approach to communally owned coding standards means anyone in the team can be an approver. This frees up experienced developers from being gatekeepers of quality while giving more challenging and engaging opportunities for growth to less experienced developers as they’re forced to contemplate broader sets of criteria their code must adhere to beyond simply “does it work?”

Automate where possible

When approaching a code review there is usually some kind of coding standard to which the team adheres to. Critically assessing code can be time consuming and laborious so any way we can reduce cognitive load should be explored.

Simply put, if we can automate part of the review process effectively then we should. This could include things like maximum line length, indentation size, spaces after function name etc… Abstracting away these minor aspects of code style means reviewers can focus their attention on higher level concerns.

This can be accomplished using linters like eslint. Code formatters can be implemented as a pre-commit hook or even integrated with the developers IDE so that when they save a file the code is automatically formatted - this means that we can have linter config files as part of the codebase so that the same formatting rules are always applied regardless of who has checked out the code.

Automation isn’t just for linting though, we can use static analysis tools to catch missing code coverage or any security vulnerabilities that our changes to the project dependencies might miss out on.

Automation is one of the simplest ways to bake quality into our development process for a relatively modest initial investment.

Empathetic feedback

The process of giving feedback, let alone technical feedback is at times poorly understood.

When giving feedback of any kind there are a couple of things to keep in mind:

• Am I acknowledging the subjectivity of my feedback? Eg. making conclusions without room for disagreement
• Have I left my own biases and preferences at the door?
• Have I thought about how the person who asked might receive my feedback?
• Are my critiques actionable?

This stuff all seems basic but too often it’s missing completely. Ineffective developers may seize on code reviews as a chance to exert undue influence over their peers, requiring changes that have no material impact on the outcome.

The best developers use code reviews as teachable moments, where the problem domain is explored in partnership with the person who asked for the review, if indeed something needs to be fixed. The experience or title gap between them is immaterial as they both explore what solution fits best. One-to-one interaction is preferred over review comments where some of the nuance of the conversation may be lost.

Finally, savvy reviewers will also understand that not all code quality issues need to be addressed at the code review stage. If something is unusual and perhaps a little contentious, then it can be addressed asynchronously the next time devs get together to talk about code. Good devs appreciate the cost involved in delay and will never seek to hold up code from being merged any more than strictly necessary.

This neatly leads us to the final point in how we can have more effective code reviews.

Technical feedback

One of the most common memes when it comes to developers is that it’s hard to get a straight answer from them. Every decision seems to prompt a lot of discussions about trade offs and opportunity costs.

Unfortunately I think there’s an element of truth to this. Most decisions, especially technical ones, are rarely straightforward and clear cut and usually there is an element of compromise to them. In spite of this ambiguity I think there is some general guidance we can apply when reviewing code for frequently deployed systems that have a low cost of change.

Let’s look at some valid justifications for rejecting a pull request and asking the dev that submitted it to look at it again (for brevity we’re going to assume the project builds successfully and unit tests are passing)

It doesn’t adhere to the teams coding standards

This is pretty straightforward. I won’t go into detail as we’ve already covered this but a team should agree what good looks like and agree to follow those standards. This should be the least contentious type of review comment as everyone had a chance to give their input in shaping these standards.

The code has quantitative deficiencies

This is where code quality is compromised in ways that, while not explicitly caught by coding standards, have a clear and identifiable negative impact.

For example, say a dev introduces a new dependency to the project to perform some functionality that is already present in language used. In the Javascript programming language (and others) concise dependency trees lessens potential version conflicts when upgrading. It also creates a smaller vector of attack as fewer dependencies means fewer packages that could, in the course of time, become outdated and insecure.

In this instance, rejecting this code change upholds a facet of code quality that can’t be strongly argued against, especially since the introduced functionality is already present within our chosen language.

Unjustified code duplication, using deprecated language features or indeed any change that meaningfully compromises the non functional requirements of our system are all examples of quantitative deficiencies.

Beyond these two cases I think review comments fall into the realm of subjectivity. Responsible reviewers can and should acknowledge this and frame their critiques accordingly. Teams that can compromise where appropriate show maturity, even if there are some minor disagreements initially.

Conclusion:

Doing code reviews well requires trust, openness and empathy within the teams that practice them.

Far from being a mundane technical practice, code reviews have the chance to energise and engage software delivery teams by fully placing ownship of the software they write in their hands.

So let’s democratise code quality and build a vision of how our system could work that includes everyone.

So long as they are not too simplified. To actually work, the models need to be clear, reasonably complete, and actionable. If I have a concrete question, even as simple as a decision between two alternative approaches, the model needs to give me clear guidance on how to proceed. Case in point, the testing pyramid:

Image source: https://betterprogramming.pub/the-test-pyramid-80d77535573

We’ve all seen this picture. It is a seemingly helpful guide: To minimise cost of testing and maximise reliability and resulting quality, put more effort into, and rely more on tests closer to the implementation than the tests that are covering a wider scope. Seems intuitively correct, but fails my criteria: It’s unclear what integration tests and even unit tests actually are, specifically. It also doesn’t help answer questions like “Should I run end to end tests on every pull request?”. And where does static analysis fit in? – it’s unclear, incomplete, and it isn’t actionable.

As a consequence, people don’t seem to find it particularly helpful, and even propose various modifications which resonate with their experience better, like the testing trophy, proposed by Kent C. Dodds. I think these variants come down to the misalignment about what the individual layers of the pyramid are, what value they bring, and how costly they are to execute. And I’ve seen one too many testing strategies in which the pyramid is completely upside down, but the engineering team just doesn’t know how to do their testing any better. The pyramid is supposed to guide them, alas it stays quiet.

I don’t think there’s anything wrong with the pyramid. It’s just not quite detailed enough to be useful. Good first approximation, but I think we can do better.

Some Assumptions

To make a better version of the pyramid, I will need to make some assumptions about the kind of system we’re testing, and the high-level ways of working of the team, and roles involved in the building of the product. At a high level, I will assume that:

• We’re building a distributed system, composed of independently deployed services and applications. Running the entire system requires an environment.
• Our system has external dependencies which we don’t control - services delivered by separate organisations or other, independent teams in our organisation
• We’re doing continuous delivery, even continuous deployment to production, and therefore heavily rely on a Continuous Integration (CI) service
• Our branching workflow resembles Github flow - we have a single main branch, into which contributions are made in the form of small pull requests which get reviewed before merging

This feels like a pretty typical situation most digital product teams find themselves in. The situation is probably different for game developers, desktop software developers, machine learning engineers and data scientists, and others. So, as always, your mileage may vary.

The improved pyramid

With that out of the way, here’s a (hopefully) more complete and useful pyramid:

This one has five layers, which are defined by what gets tested, where, when, why and how. The layers still form a pyramid, because in order for the upper layers of the pyramid to pass (or even run), the lower layers have to have passed first. Upper layer tests require more setup and more infrastructure, which makes them much more expensive to run than lower layer ones, so the overall volume of testing on the upper layers needs to be lower to maintain a stable cost/benefit ratio across layers. Or, from a more outcome oriented perspective:

The goal of each layer is to more cheaply and more regularly gain a reasonably high confidence that the layer above will pass, so that it does not need to be run as often

In other words, the lower layers are an optimisation of executing the upper layers, in order to reduce their cost while maintaining a high enough overall confidence in the system.

Changing tests with implementation is fine, top-heavy pyramids are not fine

Some will argue that pushing tests lower, closer to the implementation will lead to us having to regularly change tests when we change implementation, and that the execution time saved will instead be spent on keeping the lower layer tests up to date. To that, I will say that the idea that it should be possible to change implementation without changing tests is just… nonsense.

If it were true, it would mean that everything can be exhaustively tested through the interfaces it is consumed through - through the UI or public API, or something very close to it. That thinking obviously completely ignores the “physics” of software, and all the reasons we are building systems in a decomposed, modular way, which encapsulates and reuses logic. The sheer number of test cases necessary to capture all the nuances of all our business logic all at once is just not practical. And it’s not difficult to show that.

Let’s say each logical “unit” of our software requires, on average, 10 scenarios to test it thoroughly. Then, if these units are in any way dependent on one another (either one uses the other, or it uses the resulting state the other created), the number of scenarios multiply, and there are a 100 scenarios to cover for just two units, a 1000 for three, … This clearly doesn’t scale even a little bit! Let alone if our system is a complex sequential user journey, like an onboarding flow or a checkout in an online shop. That’s why we decompose software into functions and modules in the first place.

No other discipline building even remotely complicated things approaches testing in that way. Nobody sane would argue that a car should be tested only through driving it, on an actual public road, because otherwise we’d need to change the tooling when we change components. Of course the engine is tested independently from the tyres and the chassis. Therefore, the only remaining question is - how big is a unit we will test. And the bigger it is, the more tests are needed to cover it. And the more tests will need to be written again, if the unit is changed substantially. This is an inevitable reality driven purely by the complexity of the system, and doesn’t mean we shouldn’t build instrumentation for its constituent parts, just because we might replace them.

External dependencies are costly

You might also argue that layers 2 and 3 seem like an artificial split. We should be able to test everything on layer 3, surely. And in an ideal world, we could. The problem with testing against non-production environments of external dependencies is purely practical:

• They are not always available
• They are often slow or rate limited
• They lack test data or require creating it for every run
• They are stateful

All these reasons make repeatable, reliable testing solely against external systems difficult. But fully relying on mocks is not a reliable strategy either. We need to be sure that the mocks still behave like the actual system. We need to do both.

Guiding principles

So, we now have a more complete version of the pyramid. To make it an actionable model, we just need some overall guiding principles:

1. All layers of the pyramid are necessary in order to achieve a high confidence in your system.
2. Always prefer testing functionality on lowest possible layer.
3. When a layer of testing starts slowing you down, reduce the volume of that layer and replace it by coverage on the lower layers. Introduce new forms of testing on lower layers.
4. Do not give in to the temptation of moving layers “left” in the development process, executing the more expensive ones earlier or more often.

The last principle is worth talking about a little more. It suggests, for example, that running automated, system level, end to end test on each pull requests, as tempting as it sounds, is not worth the effort and complexity. End to end tests require an environment to run in (see assumptions), which would need to be created and destroyed (or allocated and cleaned up) for each pull request. In my experience, this is really complex, slow and highly unlikely to pay off in additional confidence gained, compared to running the end to end test(s) on the main branch, after merging the pull request.

With these principles, I hope the model is now actually useful - clear, complete and actionable. It should be possible to take every form of testing you can think of, decide what layer it belongs to, and therefore where and when it should execute.

Building up this pyramid will take time, and at first, expanding the higher layers will pay off much more than the lower layers. But these are diminishing returns. That’s why so many teams come up with an upside down pyramid that takes hours to run and still lets many bugs through. It’s so tempting to just add another end to end test. Don’t fall into that trap. Follow the above principles, and turn the pyramid the right way up.

Built to be testable

I have one final observation about all this: If, despite your best effort, you can’t seem to push test coverage down the pyramid, you may need to revisit your architecture, or even technology choices. Testability entirely depends of how your system is built.

In a system without stubs of external dependencies, achieving reasonably high coverage with automated end to end tests will be difficult and slow. In a system with services sharing state (a database for example), pushing tests from layer 2 down to layer 1 will be difficult. In a codebase with no concept of dependency injection (even a really simple one), moving coverage from functional tests to unit tests will be difficult (and require extensive, messy mocking and stubbing). In a system without contract testing or shared API types ensuring client/server alignment on layer 0, a lot more layer 3 tests will be required to gain the same level of confidence. In a dynamic language with few static guarantees, layer 1 will almost certainly be much larger than layer 0.

I could keep going, but you see the point. Fast reliable testing begins with smart engineering. Hopefully this more specific pyramid will help you make smarter choices.

We recently worked with a client who asked us to create a freestyle web canvas, for adding various types of ‘Card’. We love a challenge so got to work on the requirements:

• Panning and Zooming
• Add Cards from a pop up tray
• Cards should not overlap
• Drag Cards on the canvas

Too long didn’t read (TLDR)

This was a complex epic, and there was definitely some head scratching. However we finished it on time and the end result is beautiful, responsive and fully covered by end to end tests. We used DndKit for dragging and dropping, D3 Zoom for panning and zooming and Cypress for the tests, which were all a pleasure to work with, and we would do so again.

Panning and Zooming

A very stripped down version of the Canvas component is shown below, with the code required to hook up to D3 Zoom, and to apply the tranform.

The transform is applied to the canvas element, and the browser will automatically apply the transform to anything under it in the DOM tree, so nothing on the canvas needs to do anything.

export const Canvas = ({ children }: CanvasProps) => {
  const canvasRef = useRef<HTMLDivElement | null>(null);

  // store the current transform from d3
  const [transform, setTransform] = useState(d3.zoomIdentity);

  // update the transform when d3 zoom notifies of a change
  const updateTransform = ({ transform }: { transform: d3.ZoomTransform }) => {
    setTransform(transform);
  };

  // create the d3 zoom object, and useMemo to retain it for rerenders
  const zoomBehavior = useMemo(() => d3.zoom<HTMLDivElement, unknown>(), []);

  useLayoutEffect(() => {
    if (!canvasRef.current) return;

    // get transform changed notifications from d3 zoom
    zoomBehavior.on("zoom", updateTransform);

    // attach d3 zoom to the canvas div element, which will handle
    // mousewheel and drag events automatically for pan / zoom
    return d3
      .select<HTMLDivElement, unknown>(canvasRef.current)
      .call(zoomBehavior);
  }, [zoomBehavior, canvasRef]);

  // animated Zoom In, which can be called from a button event (not shown in this example)
  const zoomIn = () => {
    d3.transition()?.call(zoomBehavior.scaleBy, 1.5);
  };

  return (
    <div ref={canvasRef}>
      <div
        style={
          // apply the transform from d3
          transformOrigin: "top left",
          transform: `translate3d(${transform.x}, ${transform.y}, ${transform.k})`,
        }
      >
        {children}
      </div>
    </div>
  );
};

Drag Drop from Tray

In order to create insightful visual arrangements, users wanted to be able to see all of their cards in a tray, and to drag them from there on to the canvas. None of the DndKit examples were that close to what we wanted, so we had to strike out on our own (although the documentation is excellent, which made things easier.)

Display cards on the canvas

When an item is drag / dropped from the tray it needs to appear on the canvas, so firstly we simply hard coded a list of cards to display. These had an x, y position on the canvas, and all the information they needed to render. We sized the cards to match the grid size, and used the code below to position the cards on the canvas.

<div
  css={
    position: "absolute",
    origin: "top left",
    top: `${pixelCoordinates.y}px`,
    left: `${pixelCoordinates.x}px`,
  }
>
  {children}
</div>

Allow dropping when dragged over the canvas

In our UI, the tray popped up over the canvas, and being as the canvas was a drop target, it was initially possible to drop a card on to the canvas without having first dragged it off the tray.

To fix this we created a custom strategy to work out the drop target by composing existing DndKit strategies (as recommended by the documentation).

We first check to see if the current drag position is intersecting with the tray, and if so we return that. If not we fallback to the standard DndKit behaviour. This requires us to set up the tray as a drop target, and for the drop event to check what drop target was found (and to ignore the tray if this is the target).

The code for the custom strategy is like this

const customCollisionDetectionStrategy = () => {
  return (args: {
    active: Active;
    collisionRect: ViewRect;
    droppableContainers: DroppableContainer[];
  }) => {
    if (args.active.rect.current.translated) {
      const targetScaled: ViewRect = {
        ...args.active.rect.current.translated,
      };

      const trayRect = args.droppableContainers.filter(
        (droppableContainer) => droppableContainer.id === "tray"
      );

      const intersectingTrayRect = rectIntersection({
        active: args.active,
        collisionRect: targetScaled,
        droppableContainers: trayRect,
      });

      if (intersectingTrayRect) {
        return intersectingTrayRect;
      }

      const otherRects = args.droppableContainers.filter(
        (droppableContainer) => droppableContainer.id !== "tray"
      );

      return rectIntersection({
        active: args.active,
        collisionRect: targetScaled,
        droppableContainers: otherRects,
      });
    }

    return "";
  };
};

The drag overlay / mouse cursor should size based on the currrent zoom level of the canvas (so that it displays as it will appear on the canvas)

The Canvas has a transform property (from D3), which has an x and a y property to define the panning, and a k property to define the zoom.

We already had canvas card components from earlier, but the zoom transform was being applied to the parent canvas component, so we still needed to size the drag overlay correctly.

This was achieved using the same scale transform that was applied to the canvas:

<div
    style{
        transformOrigin: 'top left',
        transform: `scale(${transform.k})`,
      }
>

Calculate canvas position of dropped cards

To work out the position on the canvas, we need to know a few things:

• The zoom level of the canvas
• The panning position of the canvas (relative to the window / viewport)
• The drop position (relative to the window / viewport)

The canvas position is then (panning position - drop position) / zoom

We already store and have access to the zoom level and the panning position of the canvas, but the drop position is a bit trickier.

The DndKit drop event gives us the delta of the drag operation, but sadly doesn’t give us the initial position of the drag. It does however allow us to attach some custom data via a ref in useDraggable, so we store getBoundingClientRect() as initialRect, and can access it in the drop event with active.data.current.initialRect. This allows us to calculate the window / viewport drop position, which then allows us to calculate the drop position on the canvas.

The full code looks like this. transform controls the pan (x, y) and zoom (k) of the canvas.

const calculateCanvasPosition = (
    initialRect: DOMRect,
    over: Over,
    delta: Translate
) =>
  scaleCoordinates(
    {
      x: initialRect.x + delta.x - (over?.rect?.offsetLeft ?? 0) - transform.x,
      y: initialRect.y + delta.y - (over?.rect?.offsetTop ?? 0) - transform.y,
    },
    transform.k
  );

const scaleCoordinates = (coords: Coordinates, scale: number): Coordinates =>
  {
    x: coords.x / scale,
    y: coords.y / scale,
  };

Snap to grid

Once we have worked out a position on the grid, snapping to a grid is trivial! We just need to decide on the grid size, and then round the coordinates to it. There is even a nice example in the DnDKit docs.

Our code looked like this

export const snapCoordinates = ({ x, y }: Coordinates): Coordinates => ({
  x: snapCoordinate(x, gridSize),
  y: snapCoordinate(y, gridSize),
});

const snapCoordinate = (value: number, gridSize: number) =>
  Math.round(value / gridSize) * gridSize;

Drag and drop

Once we have all these items in place, we can integrate with DndKit.

There is a DndContext, that DndKit uses to store all the state:

<DndContext
  sensors={sensors}
  onDragStart={handleDragStart} // stores the activeCard
  onDragMove={handleDragMove} // uses doCardsCollide (see "Cards should not overlap" later)
  onDragEnd={handleDragEnd} // uses calculateCanvasPosition, adds activeCard to children
  collisionDetection={customCollisionDetectionStrategy()}
>
  {children}
</DndContext>

Then each component on the tray can useDraggable to enable drag and drop.

export const Addable = ({ id, children }: Props) => {
  const [ref, setRef] = useState<Element | null>(null);

  const { attributes, listeners, setNodeRef } = useDraggable({
    id,
    data: { initialRect: ref?.getBoundingClientRect() },
  });

  const updateInitialRectAndForwardRef = (element: HTMLDivElement | null) => {
    setRef(element);
    setNodeRef(element);
  };

  return (
    <div ref={updateInitialRectAndForwardRef} {...listeners} {...attributes}>
      {children}
    </div>
  );
};

Drag Cards on the canvas

Once the cards are on the canvas, we can use DndKit again to make them draggable. This is a bit different to dragging / dropping from the tray, as nothing new gets added to the canvas, and instead an existing item changes position.

The DndContext is much the same as before

<DndContext
  sensors={sensors}
  onDragStart={handleDragStart} // stores the activeCard
  onDragMove={handleDragMove} // uses doCardsCollide (see "Cards should not overlap" later), updates pixelCoordinates
  onDragEnd={handleDragEnd} // updates position of activeCard
>
  {children}
</DndContext>

The cards on the canvas are slightly more complex, as they have to position themselves on the canvas, and update their position temporarily while they are being dragged.

export const Draggable = ({
  id,
  pixelCoordinates,
  k,
  children,
}: DraggableProps) => {
  const { attributes, listeners, setNodeRef, transform } = useDraggable({
    id,
    data: { pixelCoordinates, id, ownId },
  });

  return (
    <div
      // position of card on canvas
      css={
        position: "absolute",
        origin: "top left",
        top: `${pixelCoordinates.y}px`,
        left: `${pixelCoordinates.x}px`,
      }
      // temporary change to this position when dragging
      style={
        transform
          ? { transform: `translate3d(${transform.x}, ${transform.y}, 0)` }
          : {}
      }
      ref={setNodeRef}
      {...listeners}
      {...attributes}
    >
      {children}
    </div>
  );
};

Cards should not overlap

One of the requirements was that cards should not overlap on the canvas, so we needed to detect when collissions would occur and prevent them.

There are two collision detection scenarios, when drag dropping from the tray, and when dragging around the canvas. The 2 situations are very similar, the main differences being that the calculation of the canvas position is different when dropping from the tray, and a card being dragged around the canvas doesn’t need to worry about colliding with itself.

The cards themselves are square, so the code to detect whether two cards collide is trivial. The one minor complication is that the collission detection has to take place after the coordinates are snapped to the grid.

const doCardsCollide = (card1: Coordinates, card2: Coordinates) =>
  Math.abs(card1.x - card2.x) < cardSize &&
  Math.abs(card1.y - card2.y) < cardSize;

When dragging, if a card on the canvas would collide, we add a red overlay to it. When dragging around the canvas, we show the last known good position of a card with a dashed outline, which is where the card will go if it is dropped. This updates as a card is dragged, and snaps to the grid. Where a card on the canvas would cause a collission, the last known position simply stays where it is, until the dragged card is moved in to a collission free space.

Testing

We added Cypress Custom Commands, like the one below to make it easy to write end to end tests.

The wrap command turns a jquery object in to a cypress object (that you can then chain other cypress comannds off), and the trigger command creates simulated events. There is a slight annoyance in that there are quite a few events triggered in response to various mouse operations, but it is all encapsulated in the custom command so writing the tests is still easy. { prevSubject: 'element' } specifies that the dragOntoCanvas command can only be chained off cypress commands that yield element’s.

Cypress.Commands.add(
  "dragOntoCanvas",
  { prevSubject: "element" },
  (
    item: JQuery<HTMLElement>,
    { startCoordinates, endCoordinates }: DragOntoCanvasOptions
  ) => {
    // `force: true` shouldn't be needed, but the tests think that
    // the drag overlay is covering the canvas (which is true) and
    // that this prevents mouse operations (which is false)
    const force = { force: true };
    const leftButton = { button: 0 };

    const dragStart = { ...leftButton, ...startCoordinates };
    const dragOver = { ...endCoordinates, ...force };
    const drop = { ...leftButton, ...endCoordinates, ...force };

    const pointerEvent = { eventConstructor: "PointerEvent" };
    const mouseEvent = { eventConstructor: "MouseEvent" };
    const dragEvent = { eventConstructor: "DragEvent" };

    cy.wrap(item)
      .trigger("pointerdown", { ...pointerEvent, ...dragStart })
      .trigger("mousedown", { ...mouseEvent, ...dragStart })
      .trigger("dragstart", { ...dragEvent, ...force });

    cy.findByTestId("canvas")
      .trigger("dragover", { ...dragEvent, ...force })
      .trigger("mousemove", { ...mouseEvent, ...dragOver })
      .trigger("pointermove", { ...pointerEvent, ...dragOver });

    cy.findByTestId("canvas")
      .trigger("drop", { ...dragEvent, ...force })
      .trigger("mouseup", { ...mouseEvent, ...drop })
      .trigger("pointerup", { ...pointerEvent, ...drop });
  }
);

We can then use the custom command in tests like this.

cy.findAllByText("cb894").dragOntoCanvas({
  start: { clientX: 50, clientY: 50 },
  end: { clientX: 700, clientY: 200 },
});

Wrapping up

So there with have it! A fully tested custom web canvas that you can drag cards to, and then rearrange.

It took around one sprint to spike, and then another to get to v1, and there have been subsequent iterations to add features and improve performance, which may become topics for future posts!

If this sounds like the sort of work you would like to do then come join us, or if you are tackling a similar problem at your company please get in touch.

This tutorial is a continuation of the earlier Introduction to WebAssembly Text

If you are not familiar with WebAssembly Text (WAT), then please read the above introductory tutorial first because from this point on, I will assume that you are at least able to read and understand a WebAssembly Text program.

In the tutorials that follow, we will take a detailed look at how to implement an application in WebAssembly Text that plots the Mandelbrot and Julia Sets

But Why Not Just Write the Solution in Rust?

I did here!

But here’s the thing…

When I wrote the above solution in Rust, I enjoyed all the advantages of using a language with much richer programming constructs and a compiler that turns out almost bullet-proof code. However, when I used wasm-pack to transform the Rust executable into a .wasm module, the resulting file was 74Kb in size.

This is certainly not large, but it was much larger than I expected given the simplicity of the task being performed.

So as a matter of both curiosity and education, I set about re-implementing this program in WebAssembly Text (WAT) to see just how small I could get it.

The results are encouraging because the hand-crafted .wasm file is now about 150 times smaller - just 493 bytes…

Live Demo

Plotting Fractals Using WebAssembly Threads and Web Workers

Table of Contents

1. Plotting Fractals
2. Initial Implementation
3. Basic WAT Implementation
4. Optimised WAT Implementation
5. Plotting a Julia Set
6. Zooming In
7. WebAssembly and Web Workers