Categories
Uncategorized

Common TypeScript errors and how to fix them

At work we recently changed the builds for our project to fail when new TypeScript errors are added, so I took some time to clean up a bunch of old TS errors in our codebase, some of which have been there for over 5 years. There were close to 600 errors when I started (down to about 30 now), and I’ve found that most of them fall into a few general categories. I’m not a TS expert, but I thought I’d write this post to explain the most common issues I’ve seen and suggest ways to solve them in case you come across similar in your own code.

Finding and reading TypeScript errors

First, let me explain how to find and read errors, because this was not obvious to me when I started working with TypeScript. Some folks may already see errors show up in their editor if you’re running the TypeScript language server, but if you want to list errors explicitly, you can run the CLI command tsc -p your-project/tsconfig.json --noEmit (you may need to prefix this command with either yarn or npm run depending on your local environment).

It’s important to note that unlike linters, the TypeScript compiler has to run on a whole project; it cannot be used to look for errors in just one file.

Reading TypeScript errors can be daunting. They are often filled with so much information that it can be very difficult to even understand what TS is complaining about, let alone how to fix it. The first thing to keep in mind is that often TS will put the actual problem somewhere near the end of the error message, so look there first.

Argument of type '{ name: string; title: string; flavor: "green"; }' is not assignable to parameter of type '{ name: string; title: string; dogs: number; flavor: "grape" | "green"; beverage: "coffee" | "tea"; }'.
  Type '{ name: string; title: string; flavor: "green"; }' is missing the following properties from type '{ name: string; title: string; dogs: number; flavor: "grape" | "green"; beverage: "coffee" | "tea"; }': dogs, beverage

The above error is saying, “you’re passing an object to a function, but the object is missing two properties: dogs and beverage“. Notice that the useful information is right at the end. I usually read TS errors from the bottom up.

Many errors come down to TypeScript saying,

  • “I don’t know what type of data this is”
  • “this object is supposed to have a property, but it’s not here”
  • “you are passing an object with a property to a function which doesn’t know about that property”
  • “this variable or property is supposed to be of type X, but you’ve provided a value of a different type”. (This last one can often be the most confusing, because types themselves can sometimes be really complex.)

Let’s look at some specific examples.

Handling falsy variables

Probably the most common error I see is something like this:

const username = getUsername();
doSomethingWithUsername( username ); // Error: Argument of type 'string | undefined' is not assignable to parameter of type 'string'. Type 'undefined' is not assignable to type 'string'.

This happens when the data comes from a place (in this case, getUsername()) that has a chance to fail (it might return a falsy value like undefined or null) and you’re passing that data to a function (here, doSomethingWithUsername()) which requires a non-falsy value (here, a string).

In this case, we can solve the error by handling the failure case. Sometimes that means reporting the error to the user, but often we know that such failures are temporary (eg: during the first render of a React component before data has loaded) so we could simply provide a fallback:

const username = getUsername();
doSomethingWithUsername( username ?? '' );

Another common fix for that issue is to alter the function you’re calling so that it accepts falsy values and handles them itself; in this case changing the parameter type from string to string | undefined | null and then adding a guard:

function doSomethingWithUsername( username: string | undefined | null ): void {
  if ( ! username ) return;
  // ...
}

Also, depending on your context, you can add early returns or other guards to make sure that your data exists before using it:

const username = getUsername();
if ( username ) {
  doSomethingWithUsername( username );
}
// or...
const user = username ? getUser( username ) : undefined;

Remember that falsy values are not all alike. You might know that undefined, null, false, '', and 0 are all false in some context, but TS requires you to be precise. You may see an error like this:

doSomethingWithUsername( username ); // Argument of type 'string | undefined' is not assignable to parameter of type 'string | null'. Type 'undefined' is not assignable to type 'string | null'.

That error is saying “your variable is either a string or undefined, but you are trying to pass it to a function that accepts a string or null, and null and undefined are different”.

Solving this error is usually as easy as either changing the function definition to accept more falsy types, or providing a fallback in the accepted type:

doSomethingWithUsername( username ?? null );

Related to the above, we often gloss over implicit type conversions in JavaScript that TS will not allow.

function doThing(x: string|number|undefined) {
	return x > 12; // Object is possibly 'undefined'.
}

While JS is fine with the expression undefined > 12, TS prefers you to be precise:

function doThing(x: string|number|undefined) {
	return x && x > 12;
}

And keep in mind that binary shortcuts like && and || don’t always do what we assume. We can usually consider them to return a boolean but they may not. Notice what happens if we put a return type on this function.

function doThing(x: string | number | undefined): boolean {
	 return x && x > 12; // Type 'string | number | boolean | undefined' is not assignable to type 'boolean'. Type 'undefined' is not assignable to type 'boolean'.
}

To fix this you may have to explicitly cast your implied boolean value to an actual boolean.

function doThing(x: string | number | undefined): boolean {
	 return Boolean( x && x > 12 );
}

Another example of using non-booleans as booleans is the Array.prototype.filter function. A common way to remove falsy values from an array is to filter it through the Boolean function (or an identity function), but for some reason TypeScript isn’t smart enough to know that this removes the falsy values.

const data = ['one', null, 'two'];

data
.filter( Boolean )
.forEach( ( x: string ) => console.log( x ) ) // Argument of type '(x: string) => void' is not assignable to parameter of type '(value: string | null, index: number, array: (string | null)[]) => void'. Types of parameters 'x' and 'value' are incompatible. Type 'string | null' is not assignable to type 'string'. Type 'null' is not assignable to type 'string'.

Instead we need a type guard (see more on them below) to explicitly tell TypeScript that the falsy values are being removed. We have one in our project called isValueTruthy.

import { isValueTruthy } from '@automattic/wpcom-checkout';

const data = ['one', null, 'two'];

data
.filter( isValueTruthy )
.forEach( ( x: string ) => console.log( x ) )

Something is not a string

Another common error I saw is when a React component expects a prop to be a string, and someone provides a React element instead.

function NameBlock({name}: {name: string}) {
    return <div>Name: {name}</div>;
}

function MyParent() {
    return <NameBlock name={ <span>Person</span> }/>; // Error: Type 'Element' is not assignable to type 'string'.
}

In this case, it’s actually ok to for the variable to be a React component, even though the original author may not have considered that. If the variable is going be rendered as a React child, we can change the function definition to accept ReactNode instead, which includes both strings and components.

import type { ReactNode } from 'react';

function NameBlock({name}: {name: ReactNode}) {
    return <div>Name: {name}</div>;
}

function MyParent() {
    return <NameBlock name={ <span>Person</span> }/>;
}

A variation of this issue which comes up sometimes is when you have a number that you want to pass to something that is looking for a string, or vice versa.

function doSomething( phone: number ) { /* ... */ }
const userInput = '1234567890';
doSomething( userInput ); // Argument of type 'string' is not assignable to parameter of type 'number'.

In that case you can usually change the type explicitly by using functions like String() and parseInt().

function doSomething( phone: number ) { /* ... */ }
const userInput = '1234567890';
doSomething( parseInt( userInput, 10 ) );

Incorrect inference from JavaScript

TypeScript tries its best to guess the types of code that’s in an imported JavaScript file. Sometimes it gets this wrong. A pretty common instance of this is when a function has optional parameters.

// In a JS file:
function doSomethingOptional( name ) {
	if (name) { console.log( name ) }
}

// In a TS file:
doSomethingOptional(); // Expected 1 arguments, but got 0.

In this case, we need to convince TS that the parameter is optional. One quick way to do this is to set a default value.

// In a JS file:
function doSomethingOptional( name = '' ) {
	if (name) { console.log( name ) }
}

// In a TS file:
doSomethingOptional();

Another way to do this is to use JSDoc to define the types, a technique that is nearly as powerful as using TypeScript directly. To mark a variable as optional in JSDoc, put square brackets around its name.

// In a JS file:
/**
 * @param {string|undefined} [name] the name
 */
function doSomethingOptional( name ) {
	if (name) { console.log(name) }
}

// In a TS file:
doSomethingOptional();

A second common problem is when JSDoc has already been used but isn’t very specific.

// In a JS file:
/**
 * @return {object} something probably
 */
export function doSomething() { /* ... */ }

// In a TS file:
const data = doSomething();
data.name = 'person'; // Property 'name' does not exist on type 'object'.

The solution here is to improve the JSDoc. Again, you can use most TS syntax (although if you run into a problem you can also use Google Closure syntax, like using Object.<string, string> instead of Record<string, string>).

// In a JS file:
/**
 * @return {{name: string}} something probably
 */
export function doSomething() { /* ... */ }

// In a TS file:
const data = doSomething();
data.name = 'person';

You can also import and use types from other files by using import() within JSDoc.

// In a JS file:
/**
 * @return {import('./my-types-file').DataWithName} something probably
 */
export function doSomething() { /* ... */ }

// In a TS file:
const data = doSomething();
data.name = 'person';

Finally, it’s worth mentioning that TypeScript can also infer types (in most cases) from React PropTypes, if they’re set on a non-TS React component. So if you see an error about some prop type being wrong on a JS component used in a TS file, check the PropTypes of the component!

Using Array.prototype.includes on constant arrays

There’s a bunch of code in our project that defines arrays of strings as constants. That’s very helpful for knowing what values are allowed, but it makes it difficult to use includes() on the array.

const NUMBERS = <const>[
	'one',
	'two',
];

function isValid(num: string): boolean {
	return NUMBERS.includes( num ); // Argument of type 'string' is not assignable to parameter of type '"one" | "two"'.
}

Basically TS is saying, “we don’t know if your random string is in the array”, which is of course the whole point of using includes(). This is hotly discussed in TS circles, apparently, but for our purposes we have to convince TS that it’s ok to do, which means either claiming that the variable is part of the array or that the array is made of strings. Here’s the latter solution:

const NUMBERS = <const>[
	'one',
	'two',
];

function isValid(num: string): boolean {
	return ( NUMBERS as ReadonlyArray< string > ).includes( num );
}

Using objects as dictionaries

In JS we use objects for all sorts of purposes, and one popular technique is to use an object as a dictionary of key/value pairs where we don’t know the keys ahead of time.

const data = {};

function addValue( key: string, value: string ) {
	data[ key ] = value; // Element implicitly has an 'any' type because expression of type 'string' can't be used to index type '{}'. No index signature with a parameter of type 'string' was found on type '{}'.

}

You have to explicitly tell TypeScript that the object is going to be used as a dictionary, which it calls a Record. You also have to specify the types of the object’s keys and values using “generics” (see more about these below); the first generic in Record is the key type (string) and the second is the value type (also string), so the type becomes Record<string, string>.

const data: Record< string, string > = {};

function addValue( key: string, value: string ) {
	data[ key ] = value;
}

Key types are nearly always strings, but if you don’t know what the value type is going to be, you can usually use the special type unknown. The downside is that you may have to further clarify the type later on, but it’s safe to do so (and much better than using the any type which you should probably avoid because it defeats the purpose of using TypeScript).

const data: Record< string, unknown > = {};

function addValue( key: string, value: string ) {
	data[ key ] = value;
}

I don’t know the type

There’s a big category of JS code that operates on any sort of data, which is often hard to express in TypeScript. There’s a lot to say about this, but for now we’ll just examine a very simple case, the identity function. As pure JavaScript, it will cause a type error.

function identity( value ) { // Error: Parameter 'value' implicitly has an 'any' type.
	return value;
}

But how can we add types for this, when we don’t know what the parameter is going to be? As mentioned above, you may be tempted to use the type any, but this is a trap as it opts out of TypeScript altogether. There’s a better way: this is precisely what “generics” are for. You can use them as variables within TypeScript code, separate from JavaScript variables. Instead of numbers and strings, they hold types.

We know that the return type of the identity function is the same type as its argument, so we can create a variable for that type – a generic – and then use that same variable as the return type. Generics are often single capitalized letters, but they can be words just like JS variables.

function identity< T >( value: T ): T {
	return value;
}

Here we’re creating a new generic called T. We then say that the type of the function’s argument is of that (variable) type. This allows us to say that the function returns a value of that same type. We’ve fully typed the function without even knowing what the data is! In most cases, TS can infer the types of generics from their call sites.

identity( 'hello' ); // TypeScript automatically infers that T is a string here.
identity< number >( 25 ); // Or you can explicitly set the value of the generic.

identity< number >( 'hello' ); // Error: Argument of type 'string' is not assignable to parameter of type 'number'.

The signature of the Record type described above (simplified) is Record< Key, Value >. There are other “Utility types” which can do all sorts of things; check them out!

Another common example of using a generic is the React useState hook. The hook can store and return data of any kind, so how does it know the type of data it holds? It defines a generic.

const [ state, setState ] = useState( 'value' ); // TypeScript infers that state is a string.
setState( 'other value' );

const [ state, setState ] = useState< number >(); // You can explicitly set the value of the generic.
setState( 'other value' ); // Error: Argument of type '"other value"' is not assignable to parameter of type 'SetStateAction<number | undefined>'.

Of course, sometimes you really have no idea what’s in a variable and you don’t want to make any assertions about it, such as the data returned from an HTTP API call. For those cases, you can use unknown, which forces any code that tries to use that variable to first verify its type.

function getData(): unknown { /* ... */ }

const data = getData();
console.log( data.name ); // Error: Object is of type 'unknown'.

There’s a bunch of ways to assign proper types to unknown variables, usually called “type guards” (worth reading about for other reasons!). Unfortunately asserting unknown is not as easy as it could be.

Here’s one way to resolve the error by using the as keyword, but be very careful with this technique because you can easily use it to lie to TS and cause bugs. This keyword tells TS to change the type of a variable. Notice that I told TS the data might be undefined and I have an if statement to guard against the assumption I’ve made.

function getData(): unknown { /* ... */ }

const rawData = getData();
const data = rawData as { name: string } | undefined;
if ( data && data.name ) {
	console.log( data.name );
}

Here’s a similar version which uses typeof and an inline as. Always consider the JS code that will be produced when compiled. In this example, if typeof data is “object”, and it is not falsy (null is also an object), we can safely evaluate data.name even if we are wrong about the type we put on the right hand side of as. Use this powerful keyword with care.

function getData(): unknown { /* ... */ }

const data = getData();
if ( data && typeof data === 'object' ) {
	console.log( ( data as { name: string } ).name );
}

Optional component props

Sometimes an author will create a React component with typed properties, but forget to specify which ones might be optional. This happens a lot when converting from PropTypes to TypeScript because PropTypes are optional by default. It’s also very common on boolean types because we assume that false will be the default (undefined will be the default, which is falsy but not the same).

function MyComponent(
  { name, isImportant }: { name: string, isImportant: boolean }
) { /* ... */ }

function MyParent() {
    return <MyComponent name="human" />; // Property 'isImportant' is missing in type '{ name: string; }' but required in type '{ name: string; isImportant: boolean; }'.
}

The solution here is to change the prop type to be optional by appending a question mark.

function MyComponent(
  { name, isImportant }: { name: string, isImportant?: boolean }
) { /* ... */ }

function MyParent() {
    return <MyComponent name="human" />;
}

Types are powerful but unforgiving

In many TypeScript projects it’s still possible to create non-TypeScript files, so if you’re feeling uncomfortable making a TS file, just create a JS file instead. However, even if you do, you may want to keep the types of your data in mind and perhaps add JSDoc comments to help your future self.

In general, we always write code with types in mind, even if they’re unconscious assumptions; TypeScript is merely a means of making our assumptions explicit. Often that can be annoying because it forces us to drag them out of our thoughts, deal with contradictions, and figure out what syntax to use to coerce those assumptions into code, when all we really want is to add a feature or fix a bug.

The trade-off, however, is that many mistakes we might otherwise make are rendered impossible. Perhaps more importantly, no other developer (nor our future selves) will ever be able to accidentally break our assumptions. Using TypeScript is a document of how our code is meant to be used and a guard around the logic we write to be sure that it is used as intended. TypeScript requires you to be precise, but it pays you back with stability. I hope it serves you well.

Photo by Randy Fath on Unsplash

Categories
Uncategorized

The Hunted Queen

As in the before times, this year I wrote a D&D one-shot adventure to share with my co-workers at Automattic. Sadly, I was only able to run the adventure for a few people because 2020. Still, it was a fun adventure and I wanted to share it with the internet, so here you go. (It’s based loosely on a certain scene from the Witcher.)

Queen Kalia of Bragod was married for her beauty, but has become an unexpected champion of the common people. This has made her many enemies in the court, and some whisper that the king is among them. As she returns from a distant voyage, she learns that her life may be in danger. Can you protect the queen from certain doom?

Categories
Uncategorized

grepdef: a quick way to search for definitions in code

I live inside my code editor all day and something that I need to do pretty often is to search for the definition of a symbol (a variable, constant, or function). For example, I might be debugging a problem in a JavaScript file when I come across this code:

//...
let newData = translateDataFromRaw(rawData);
//...

I need to see translateDataFromRaw to understand how it works. How can I do this?

If I’m using an IDE or a project covered by a tags file, I can use the magical “jump to definition” feature. This is usually the best option as it is fast and nearly always accurate. However, if I’m using a simple code editor that does not have a “jump to definition” feature, or if the feature is still indexing my project (this can take quite a while sometimes), or if I want to find that function on the command-line for some other purpose without having to open up an IDE, then this will not work. What are my other options?

I could use grep to search my files for the string translateDataFromRaw and then find the definition from among the results. This works well and is pretty simple, unless that function is used in a lot of places. If there are hundreds of results, it may take me forever to find the one I want.

To help with this situation, I wrote a little command-line tool called grepdef. With it, I could just run grepdef translateDataFromRaw and I’ll instantly get the grep result for the definition.

$ grepdef translateDataFromRaw
./src/translators.js:function translateDataFromRaw(data) {

It’s way faster than “jump to definition” on some of the projects I work with. I even have it integrated into my editors with the plugins vim-grepdef and vscode-grepdef.

How does it work?

It’s basically the same as the grep technique I described above, except that it has two advantages:

First, it uses a regular expression to find symbol definitions unique to a specific code language. This can definitely be inaccurate, but in my experience it’s close enough.

Second, it uses ripgrep which is blazingly fast.

All together, grepdef saves me hours of development time. Maybe it will help you too?

Right now it only supports two language types: PHP and JavaScript/TypeScript, but it’s built to be easily extensible to other languages. Please suggest them as issues!

(Photo by Clay Banks on Unsplash)

Categories
Uncategorized

Unusual Things

Every year I write a D&D one-shot game for my friends and co-workers at the Automattic Grand Meetup. This year’s adventure was titled “Unusual Things” and has a certain… upside down feeling to it. I hope you like it! Here’s the blurb:

Nothing much happens in the mountaintop town of Hawkurns, where the populace mines magical crystals and their kids to get into all kinds of mischief. Recently, however, people have been disappearing, and no one knows why. Rumors of unusual things are everywhere. Can you help solve the mystery before something worse assails this small town?

If you’d like to explore the adventure yourself, click here and download the PDF! It runs in just about three hours and is designed for 4-7 characters of level 2.

Categories
Uncategorized

What I learned writing a game engine

One of the things I did on my recent sabbatical was to start coding a video game in JavaScript. I learned a lot in the process, and I thought I’d write down some of my experiences. (For the record, while I wrote the game engine myself, I used the excellent Pixi library for graphics.)

Classes are useful after all

The more I’ve written code in JS the more I’ve eschewed classes. JavaScript’s primitive object type is pretty succinct and powerful already. Also, the prototype object model has an awkward syntax when using new. The class keyword makes this better, but can increase confusion due to call site binding (class fields with arrow functions fix this, but that feature is still in stage 3). It doesn’t help that the new keyword is misleading to those familiar with classical inheritance. Lastly, using class makes it easier to use inheritance instead of composition.

I don’t actually think of any of these as problems with the language, since I find it easy to write clear code with just functions that pass around data. In my experience, web applications do best when they never mention the new keyword at all. However, my first attempt to apply this methodology to game development ended in quite a mess.

It turns out that while most of the web application data I’ve dealt with is easily represented and moved around using primitive types, the entities in a video game have so much internal state and so much interactive functionality that a one-way flow approach (like the one popularized by React) is terribly inefficient. The way I did it, rendering each frame was a nightmare of indirection.

Keeping the state and functions together within each object and sharing those functions using inheritance ended up being a whole lot simpler. It also meant that I had real object types that could be validated using Typescript.

Math is useful after all

When I tell people I’m a programmer, they often think it means I’m great at math. I’m not great at math; that’s why I like computers so much! But it turns out that dealing with virtual objects in a virtual space requires quite a bit of algebra, trigonometry, and calculus.

It doesn’t help that most of the game development tutorials I came across assume that you’re using an existing game engine to do the work. The first time I tried to figure out how to calculate the angle between two sprites, all I could find was instructions for slerping quaternions.

Do you know what a quaternion is, or spherical linear interpolation? Well, they’re totally unnecessary in this case. In 2d space, all I really needed was arctan2(), which I probably learned about in high school and subsequently forgot. But it took me a while to figure that out. I think that if I had been taught math in school in the context of game development, I probably would have paid a lot more attention.

Making a scripting language is great

As I was developing events in the game, I wanted a way to easily centralize dialog such that I didn’t need to write code to create conversation trees (actually they’re graphs!). I did this by creating a JSON file that encoded dialog nodes, each with their own responses, and links between those responses and other nodes.

This worked really well, but I quickly realized that I needed to also encode functions into these nodes so that actions could be taken when certain choices were made. When a player insulted the captain of an enemy ship, for example, I wanted to decrease the happiness of that character and possibly have the enemy ship attack. Furthermore, I found that when a ship did attack, I wanted to encode the ship’s behavior itself. In short, I began to write the content of my game in a script.

At that time I was putting JavaScript functions into my JSON file to encode actions. That meant that it wasn’t really JSON anymore. If I wanted to serialize my nodes, I’d have to put those functions into strings. This immediately felt wrong because I’d have to let my game engine execute arbitrary JavaScript, which is about as big of an anti-pattern as you can get.

Professional game engines are not written in JavaScript, so they can use JS as a scripting language, but I would have to do something else if I wanted to allow executing arbitrary scripts. I began to wonder if I could write my own, very simple scripting language for the purpose. I was fortunate to discover Nearley which made that both possible and fast.

I first wrote a grammar that looked something like JavaScript, except without any variables or named function definitions (although there were anonymous functions). Statements were composed of expressions which were composed of function calls and primitives. The functions were all pre-defined; this was my game’s API. All of that worked pretty well, but I ran into a problem when I added if statements for control flow.

Each statement had to end with a semicolon. Since the language’s programs had to be encoded as a JSON string, and JSON strings cannot have newlines, it was imperative that I could chain statements together on one line. However, if statements in most languages do not end with a semicolon. I had a really hard time encoding this into the grammar, and so each if statement had to end in a semicolon as well. With that requirement I found conditions really awkward to write.

if (distanceToPlayer() < 5) { if (getNpcHappiness('enemy1') > 2) { gotoNode('enemy1Greeting'); }; if (getNpcHappiness('enemy1') <= 2 { attackPlayer(); }; };

Eventually I realized: I had multi-argument functions and I had anonymous function definitions… so I could encode control flow as a function call! That is, instead of this procedural condition statement:

if (distanceToPlayer() > 5) { accelerate(); };

I could write this functional version:

if(distanceToPlayer() > 5, { accelerate(); });

The difference was subtle but powerful. I was able to remove nearly half of the grammar I had written, since the function call syntax was already supported.

My scripting language is not extremely powerful, but even with a simple grammar in place I was able to write most of the events of the game entirely in one JSON file. I can see how it would be possible to write a dedicated dialog and level editor for that file, which would totally separate the game and its engine.

Conclusion

I’d love to show you my game, but I was pulled into other adventures on my sabbatical before I was able to finish writing it, and it has a long way to go. Maybe I’ll pick it up some time in the future, but for now I’m happy about all the things I learned getting it as far as I did. If you want to look at my code, feel free to browse the repo!

Categories
Uncategorized

Time Greed and the benefits of slowing down

I just came back to work from a three-month sabbatical, and I wanted to capture some of the things I learned during that time. Part of my sabbatical was a meditation retreat where an important theme was paying attention to the forces of Greed and Aversion in our minds. Over the course of the retreat and in the weeks since I’ve come to notice that a large amount of my stress is caused by a strange type of Greed that I’ve been calling “Time Greed”.

Some Definitions

I capitalize Greed and Aversion here because they represent concepts that cannot easily be simplified into single words. It may be worthwhile to describe these concepts briefly, which I will attempt below, but please be aware that even these descriptions may not quite get at the deeper meaning. If you’d like to know more, look up the concepts of the so-called “Defilements” (another problematic translation) in Buddhist psychology.

Also it’s worth being explicit that even though these words in our language have a negative connotation, in this context they are not meant to be used as judgmental labels. This is a strong habit in our culture, to label experiences as “good” and “bad”, but that is just another form of Greed and Aversion at work. The practice of wisdom is to recognize when certain forces are present in the mind and to discern, through context, if they are wholesome (useful, kind, timely) or unwholesome (not useful, kind, or timely). There’s no need to do much more. If we really truly see that they are unwholesome, they will drop away by themselves, just like you’d drop a hot coal.

Greed, then, is characterized by the mental pull toward some object, thought, or experience. It usually manifests in the body as physical tension and in the mind as a strong desire. This can be anything from the desire for a sandwich to the desire for a different government and also includes very subtle desires like wanting someone to walk faster on the sidewalk. Taking actions when that pull is the motivation can sometimes cause results that end up causing even more problems, for ourselves or others.

The concept of Aversion is similar to Greed, but rather than being a pull toward some object or experience, it represents a feeling of pushing away. We notice this when we want something to stop or to change or to be fixed. That “I have to fix this” thought is very pervasive for me. It’s probably not too surprising to say that acting out of Aversion (sometimes even translated as Hate) can have consequences that are not helpful for anyone.

We can, however, want things (or want to change things) without Greed or Aversion being present. Altruistic, wholesome desires, possibly for the same exact things that trigger Greed and Aversion, do not have that same tension or mental pull. We can want something to happen but not become angry when it does not. It happens all the time. I’d like to pet that dog up ahead, but the dog turns off on a different street. I’d like to eat Indian food, but the restaurant is full and I have to get pizza instead. It’s not a problem. Desire, therefore, is not a problem. It’s our relationship to that desire that causes stress. Greed appears when a desire is held so tightly that it becomes a requirement for our happiness.

Time Greed

With those definitions out of the way, I can explain a bit about what I discovered. Unsurprising to me, I quickly noticed a lot of both Greed and Aversion appearing all the time in my daily life, in both gross and subtle ways. Usually the target of those forces was pretty obvious. I wanted lunch, I wanted someone to like me, I wanted an annoying noise to stop, I wanted someone to behave differently, I wanted to feel different, look different, or to have things I didn’t have. But pretty frequently I noticed a kind of pull toward something, and there didn’t really seem to be anything there.

In these moments, the only thing I could find that I was being pulled toward was a plan. What I wanted was for things to go a certain way in the near future. This usually manifested as a feeling of tension as Greed told me I needed to prepare for, or act on, the next step in my (often unacknowledged) agenda. “Time is running short! Hurry! Get on with it!”, the mind would say. I started to call this feeling, “Time Greed”, and the more I investigated, the more of it I found.

The neat thing about this practice is how little effort it takes. Noticing the feeling of Greed is just a habit of remembering. It’s not always easy to remember, but when I feel that tension, all I really have to do is acknowledge it, and maybe look to see what the target is about. Often, for the less tangled desires, just that investigation itself releases the stress. I see that I don’t really need that sandwich as much as I think I do, and the body and mind relax all by themselves. This has worked too, with Time Greed. I see that I probably won’t be fired if I am a few minutes late to work, and the stress just evaporates. Poof!

Naturally there are some times that I do need to hurry, but they don’t seem to be as often as Greed would have me believe. It’s been a longstanding mantra of mine when I find myself rushing to get a task done, and fumbling at every step, that “if you’re in a rush, slow down to speed up”. Now I can appreciate one of the underpinnings of that saying. Stress does not improve speed, or if it does, the risk and the suffering that come along for the ride are not worth it.

(Photo by Erik Witsoe on Unsplash)

Categories
Uncategorized

Await, there’s more!

This week I gave a talk at my local JavaScript meetup on the history, use, and future of Promises and I thought that you, dear reader, might be interested as well. Here’s the blurb:

JavaScript is an asynchronous language; it is designed to react to events and to trigger jobs that take an unknown amount of time to complete. While there is a fairly standard way to call functions when something happens, until recently there had been no standard way to chain these functions together, nor an easy way to handle failures inside the chain. Promises provide the solution. This talk will guide the group through the motivation for Promises, how they work, and what comes next (async/await and beyond).

My slides for the talk are at the following link:

https://sirbrillig.github.io/js-promises-slides/

During research for the talk I discovered that Top Level Await is already available in Chrome Devtools (not in Chrome itself). So if you’re just experimenting, you can run both of the following snippets. The first one uses the native Promise methods to fetch some data (try it!).

fetch('https://jsonplaceholder.typicode.com/todos/1') 
  .then(response => response.json()) 
  .then(json => console.log(json))

The following one is the same, but uses async/await.

const response = await fetch('https://jsonplaceholder.typicode.com/todos/1');
const json = await response.json();
console.log(json);

If you’re not familiar with the syntax of await, the new thing here is that with the proposal, you can use it outside of an asynchronous function. Otherwise, you can only use the await keyword in a function declared with async.

Writing a function with the async keyword actually just guarantees that the function will return a Promise object. If it does not, its return value is wrapped in a resolved Promise automatically. So in actual code today you’d need to write something like the following in order to use await.

async function getDataFromServer() {
  const response = await fetch('https://jsonplaceholder.typicode.com/todos/1');
  const json = await response.json();
  return json;
}

async function main() {
  console.log(await getDataFromServer());
}

main();

Just for completeness, you’ll need to know that while we can handle Promise rejections (and thrown Errors) with the catch() method on a Promise, you’ll probably want to use the regular try/catch syntax to handle rejections when using await.

When using await, if the Promise is rejected, the await expression throws the rejected value.

async function main() {
    try {
        await fillKettle();
        await boilWater();
        await addLeaves('green');
        await steepTea('1 minute'));
        drinkTea();
    } catch (error) {
        console.error('There was a problem making tea.');
    }
}

main();

If none of that makes sense to you and you’d like to start from scratch learning about Promises in JavaScript, check out the whole slide deck. A big thanks to everyone who came and especially to those who asked questions!

Categories
Uncategorized

Alternatives to Else

One of the first imperative programming concepts I ever learned was if/else. With this relatively simple power tool I could make decisions in my code based on any number of factors.

Of course, my early programs were… a little hard to read. I hadn’t yet learned one of the maxims of programming that I try to live by today: write code first for humans to read, then for computers.

Using if for conditions is pretty important. There’s other ways to code, but as a concept it’s usually extremely readable. On the other hand, the seemingly harmless else can add a world of problems. I’d like to talk a bit about why I feel this way, and explore some alternatives.

(note: these examples are in PHP, but the concepts are the same for javascript and other languages.)

Cognitive Load

The biggest issue I have with else is… “what’s the condition?” An else statement prefaces an arbitrarily large block of code, but unless I happen to have just written the code, I have to read upward past another arbitrarily large block of code before I can figure out the conditions on which that block will execute.

Many if/else statements start off with each block holding just one or two lines, and surely that’s not an issue, right? Who could complain about this innocent code?

if ($wantSomeToast) {
  butterBread();
} else {
  makeASandwich();
}

The problem arises when those code blocks grow. And they will grow. Without careful pruning, code only gets bigger. Perhaps not every if block will get to be 300 lines long, but at least some of them will, and in my opinion there’s really no need to take that risk.

When you’re debugging some issue and a diff or your editor only shows something like the following, who knows what’s really going on here?

  // ...
  bakeCake();
  $frosting = new Frosting();
  engageFroster($frosting);
} else {
  drillHole();
  $jam = Jam::fetchStrawberry();
  $result injectJam(new JamInjector(), $jam);
  if ($result !== 'ok') {
    throw new \Exception('Jam is jammed');
  }
  //...

Parsing this means scrolling up until you find the if condition, and then holding that in your head while you work on the else block. And what if there’s else if statements, or nested conditions?

Alternatives

In short, I think that when else is used it needs to be as close to the if as possible. Within a few lines, really. Because, again, this is not a problem:

if ($wantSomeToast) {
  butterBread();
} else {
  makeASandwich();
}

But this is a problem:

//... many lines...
} else {
  makeASandwich();
//... many lines...

To that end, any time you can easily avoid else, I’d argue that it’s worth the effort. Here’s some ways to do that:

The first thing I see a lot is a simple boolean return.

function isTostValid($toast): bool {
  if (empty($toast->grain)) {
    return false;
  } else {
    return true;
  }
}

I know that this models the original developer’s thought process, but look how large it is, and remember that it’s far too easy for later developers to come in and add unrelated code to those blocks. What if we did this instead?

function isTostValid($toast): bool {
  return ! empty($toast->grain);
}

Much easier to read. Another alternative is an early return. This set of conditions could grow very large after a while:

function getFavoriteJam($name): string {
  $jam = null;
  if ($name === 'joe') {
    $jam = 'marmalade';
  } elseif ($name === 'jane') {
    $jam = 'raspberry';
  } else {
    $jam = 'strawberry';
  }
  return $jam;
}

But it could be easily turned into a version without else, by using early returns. The first conditions will still override later ones, but any time we follow code and see return we know that we can stop reading. It becomes a set of simple decision making tools rather than one complex multi-decision machine.

function getFavoriteJam($name): string {
  if ($name === 'joe') {
    return 'marmalade';
  }
  if ($name === 'jane') {
    return 'raspberry';
  }
  return 'strawberry';
}

Sometimes you can turn an if/else into two if statements by repeating and inverting the first if condition. This may seem to violate the DRY (Don’t Repeat Yourself) principle, but in practice it grants a lot of clarity.

if ($name === 'joe') {
  $jam = 'marmalade';
} else {
  $jam = 'raspberry';
}

That else can just have its condition repeated to become:

if ($name === 'joe') {
  $jam = 'marmalade';
} 
if ($name !== 'joe') {
  $jam = 'raspberry';
}

Another common pattern which this demonstrates is a condition being used to set a variable. In those cases, we can use null coalescing (or logical OR in Javascript), ternaries, or even function calls to make the assignment less likely to grow out of control later. Here’s a ternary version of the above.

$jam = ($name === 'joe') ? 'marmalade' : 'raspberry';

In this version it’s immediately clear that the whole line is about assigning $jam based on a condition, and the condition is the very next thing. In the first version it would take reading five lines and then drawing some conclusions to be able to be sure that’s all that’s happening, and just imagine how hard it would be when twenty other statements get added to those blocks!

If there’s a lot of computation or multiple decisions needed before the variable assignment, put all of that into its own small function and you end up with this, which is even more clear:

$jam = getJamFlavor($name);

There’s a lot of other ways we can avoid else in our code, and even if you’re skeptical, I’d like to recommend you give it a try! There’s no one best option, and it might just change how you think about coding entirely.

(Photo by Jens Lelie on Unsplash)

Categories
Uncategorized

How do we deal with dependencies in PHP

Generally what we want to do when we’re coding something is to call a function.

do_the_thing();

While some of the functions we call could stand alone and some could be methods on an object, they’re all just function calls.

So why does most PHP code have so many classes? I don’t think code organization is a good answer, since we have access to namespaces which can do that just as well. I believe the only valid reason apart from value objects is because of trying to manage dependencies.

Let’s say you are writing code which wants to refund a purchase through a billing system. You call something like refund_purchase($purchase) and you expect to get back a receipt id for the refund. From most caller’s perspectives, that’s all we should need to worry about.

However, let’s say the refund function needs to do the following things:

  • Contact the payment processor to initiate the refund.
  • Create and save a new receipt to the database.
  • Queue webhooks for the refund.

In order to do any of those things, it requires references to other systems. What if we want to switch out those other systems? Perhaps we want to test our function without a database, or we want to change the payment processor. This is where dependency injection comes into play.

Dependencies as Arguments

In order to use dependency injection in the simplest sense, we need to pass interfaces for each of the dependent systems into our function as arguments.

function refund_purchase($purchase, $processor, $database, $webhooks) {}

This is not scalable, though, because as the number of dependencies increases, so does the number of arguments, making it very confusing to call the function in the first place. In addition, when a developer wants to add a new refund call, they also have to figure out how to get each one of the dependencies, each of which may have their own dependencies!

Wrappers

The alternative to passing dependencies as arguments is to use a wrapper function to pre-inject our dependencies before they are needed. In PHP (and other Object-Oriented languages) the most accepted way to do this is using a class constructor as that wrapper. (In some other languages we might use a Higher Order Function, but this is much more challenging in PHP).

class Refunds {
  public function __construct($processor, $database, $webhooks) {
    $this->processor = $processor;
    $this->database = $database;
    $this->webhooks = $webhooks;
  }

  public function refund_purchase($purchase) {}
}

But now we are stuck with another problem: were do we call the wrapper? Since we might want to ask for a refund in any number of places, generally we must force the calling code to call the wrapper as well.

$wrapper = new Refunds($processor, $database, $webhooks);
$wrapper->refund_purchase($purchase);

That’s not a whole lot better than putting the dependencies into the function arguments, though; it’s confusing and annoying for the caller. Also, any time we want to add new dependencies we would have to change every single caller.

Factories

A solution to the issue of where to call the wrapper is to put it into a stand-alone function.

function create_refund() {
  global $wpdb;
  $processor = new DefaultProcessor();
  $database = $wpdb;
  $webhooks = new Webhooks();
  return new Refunds($processor, $database, $webhooks);
}

The calling code must still create the wrapper before it uses the function it wants, but at least that can be broken down to a one-liner.

create_refund()->refund_purchase($purchase);

In practice, these wrapper calls are often put into static functions, and can even be functions attached to the classes they are meant to wrap, which puts them close to the function call we want and so makes them easier to discover.

Forwarding Functions

It’s also possible to wrap the wrapper call into its own forwarding function to remove any need for the caller to know about our wrappers.

function refund_purchase_with_defaults($purchase) {
  return create_refund()->refund_purchase($purchase);
}

This may seem like an extreme level of redirection, but it provides the best of both worlds: as long as the forwarding function does not do anything apart from call the wrapper and its method, most callers can ignore dependency injection entirely. This only works if we make sure never to use the forwarding functions inside other wrappers, though.

Containers

Eventually we end up with tons of wrappers, as there are many functions that have dependencies and they can’t always be grouped easily. This presents two new problems: it can be hard to locate the wrapper you need, and in order to keep everything as simple as possible for the caller, we end up creating and re-creating the wrapper class many times within a single execution of the code. This can immensely bloat our memory usage since each one of those instances keeps its own state and references.

Therefore many projects end up with a Dependency Injection Container which puts all the wrapper functions into one place and also caches the instances it creates, making it very easy for callers to keep their one-liners without having to worry about memory issues.

class Container {
  public function create_refund() {
    global $wpdb;
    if (! $this->refunds) {
      $processor = new DefaultProcessor();
      $database = $wpdb;
      $webhooks = new Webhooks();
      $this->refunds = new Refunds($processor, $database, $webhooks);
    }
    return $this->refunds;
  }
}

Having these wrappers creates a new problem for our functions themselves, though, which I mentioned above: it’s now so easy to call other wrapped functions that it’s tempting to use the wrappers to make calls to other systems rather than using injected references. Doing this defeats the purpose of our wrappers entirely and we end up right back where we started. Therefore we must have some mechanism, usually just code review, which prevents anyone from using wrappers within a wrapped function.

Do we need all this?

All of this work is built on the premise that dependency injection is good and a necessary thing to have. The counter-argument is that without that premise, our code is so much simpler. So let’s examine the concept a bit more.

In any given project there will probably be functions for which the dependencies are relatively trivial. Let’s say a function needs to log events for analysis, but we don’t want to log anything while running automated tests. Rather than injecting a logger, we could just have the logger function itself detect if we’re running inside a test and if so, do nothing. If more special cases are needed, those can be added directly to the function as well.

function log_message($message) {
  if (! defined('ARE_TESTS_RUNNING')) {
    write_log_message($message);
  }
}

This technique has the advantage that it’s more foolproof than relying on callers to substitute dependencies, so we might be doing it anyway to protect our data. Even though it relies on global variables and constants, a PHP process is bounded in both time and scope so in many projects this reliance on globals creates fewer problems than it solves.

It’s also likely that in any given project there are at least a few functions which have very complex dependencies, and therefore are hard to test without running them in isolation. These functions probably benefit from using a wrapper approach as described above.

So when exactly do we need dependency injection and when is it just needless work? Even though to me the elegance of separating dependencies is a reason in itself, to be honest I can only think of two real needs. I think that we must ask these questions of each dependency we find.

Questions to ask

  1. Is a dependency large or complex enough to require mocking during testing, or do we need to test the interaction with a dependency itself?
  2. Is it likely that a dependency will need to be changed now or in the future because it represents some system that has more than one implementation?

As soon as any dependency in a project answers “yes” to one of the above questions, I think that it’s worth including a Dependency Injection system of some kind. Even if the answers are “no”, it might be a good idea if the project is a library which itself is used by other systems, because it’s often hard to predict how a library might be used. It can be a pain to be forced by your libraries to adopt specific other libraries.

And if you do have a Dependency Injection system, if any given dependency does not answer “yes” to the above questions, then perhaps you can ignore injecting it. The purist in me rebels against the idea of a single function including both injected dependencies and non-injected dependencies, but given the large amount of code needed to use a wrapper in PHP, it may just be the better choice.

(Photo by John Carlisle on Unsplash)

Categories
Uncategorized

Legend of the Whale

As has become tradition, every year I design and run a one-shot Dungeons & Dragons game for my co-workers at our annual gathering. This year I was heavily inspired by my experiences playing Breath of the Wild and I wanted to try to re-create that particular “Zelda” feeling in D&D. I’d love to have done more but there’s only so many puzzles you can fit into a 3 hour game!

The adventure is a one-shot adventure for a group of 2nd-level characters and runs in just about 3 hours. It was designed to be fun for brand new players and experienced gamers alike. Myself and several wonderful volunteer DMs ran the game seven times over the past few weeks and I think it runs pretty smooth.

Here is the teaser text:

One hundred years ago the realm of Dunelin built four magical monuments in the shapes of giant animals to concentrate their power and defend against a demon’s attack. Sadly, they did not account for the cunning of their adversary. The demon took control of the monuments, turning them against their creators, making him nearly unstoppable.

Though it cost her life, the realm’s most powerful wizard was able to bind the demon in a magic circle, buying her people some time. All of the demon’s rage is now focused against his prison, and after a hundred years he is close to breaking free.

Time to to recover the monuments is running out.

You may download the adventure yourself here: http://bit.ly/legend-of-the-whale

Photo by Roberto Lopez on Unsplash