Copying files… sometimes

File this one under “tools that probably only I will find useful”. In the course of my normal job I need to copy files to a synchronized directory on my computer (something like a DropBox folder). The files are JavaScript code that’s been transpiled and copying them to the synchronized directory is what deploys them to my staging server.

Therefore, as I work, I need to: Code → Transpile → Deploy → Test → Repeat. My ideal work cycle is more like: Code → Test → Repeat. I want to be able to just write code and then reload my browser, not all that boring stuff in the middle.

The process of transpiling my code can be handled by a watcher (usually grunt-watch or watchify) so I don’t have to worry about that part. Unfortunately because of the deploy step my process still looks like: Code → Deploy → Test → Repeat. (Grumble grumble inefficient grumble.)

Now, one way I could handle this is just to move my whole project into the synchronized directory. That way the transpiled files are already in the right place and I don’t need a deploy step. That feels like cheating to me, though. It means that my project directory has to exactly mirror the deploy directory and it means that all of my working files are also being synchronized as I code; that’s a lot of unnecessary data over the wire.

Naturally I decided to automate my way out of this problem. I thought to myself: I could just add another task so that the watcher copies my transpiled code to the synchronized directory when it’s done! Ah, but I’d need to give it an absolute path, and several developers work on this project, each with different synchronized directories. That’s not ideal. And even worse is that another developer might want to handle the deploy process differently.

This led me to write copytotheplace. It’s a very simple library and command-line tool that will allow copying files to a directory by setting the destination as an environment variable or using a config file (or a command-line option or parameter if you’re using the library directly).

If no destination directory is set, this tool will do nothing, which allows it to be used as the last step in a build pipeline without doing anything unless specifically called-for.

To hook it into my particular build tool and watcher, I wrote grunt-copytotheplace which just loads the library into a Grunt task.

Now I just put COPYTOTHEPLACE=/path/to/my/sync/directory in the .env file in my local project directory and Grunt will copy the files there every time they change. More importantly, when other developers who don’t have that option set run their version of Grunt, nothing will be copied anywhere.

I know, it’s a weird solution to a weird problem, but it was a simple way to dramatically speed up my workflow without harming others, and so for now I consider it a win. Maybe next week I’ll come up with a better way. But in the mean time, if you find this tool useful it’s up there in the cloud for all to share. Just npm install copytotheplace and away you go! (See details in the README.)

Partial application and making tea

Partial application is like making tea. The person making the tea needs two pieces of information: what kind of tea, and how many people to serve. We can save time by knowing one of those pieces of information before we begin.

Let’s say we have a tea shop. Whenever a new person comes in with a group, you need to ask them two questions: what kind of tea would they like, and how many cups do they need? After that, you know how to do the rest. Of course, sometimes the customer hems and haws about what kind of tea so it can take a while…

But then the door opens and one of your regulars comes in. You know that she always asks for a green tea, but you’re never sure who she’s coming to meet, so you still have to ask her how many cups she’d like. Because you already know the kind of tea, you only need one piece of information instead of two. You’ve kept the first piece of information in your memory. This is partial application.

When programming, you may be writing a function that takes two (or more!) arguments. If you often use that function in a particular way, you can prepare it by creating a partially applied version that only takes one argument instead of two, keeping the missing argument in memory (often through the clever use of closures). Then when you want to call it, you can call the partially applied version.

Partial application means taking a function and partially applying it to one or more of its arguments, but not all, creating a new function in the process. (Dave Atchley)

A little harder to explain is why you’d want to do that. Well, for me the main reason is to be able to pipe the result of a series of functions together.

That is, when you have a series of functions that need to be run on the same piece of data, you might write something like:

adjustedData = adjustData( data );
preparedData = prepareData( properties, adjustedData );
doSomething( preparedData );

That works fine, but in some cases it can get messy. Also, we’ve created two temporary variables whose entire purpose is to pass the result of one function on to the next. We can do better!

doSomething( prepareData( properties, adjustData( data ) ) );

Ok, that gets rid of the variables, but it’s way less readable. Just think what it would look like if there were ten functions in that chain! What we need is a way to pipe data from one function to the next just like the | character in UNIX shells. There is a type of function which does this, sometimes called “pipe”, “compose”, or “flow”.

composedFunction = flow(
  adjustData,
  prepareData,
  doSomething
);
composedFunction( data );

Wow, that’s so much better! It’s even easy to add or remove steps in the chain. But you may have noticed that I skipped a step: prepareData takes two arguments, so how can we get that second argument in there?

What we need is a way to transform prepareData from a function that accepts two arguments into a function that accepts one argument. This is partial application.

prepareDataWithProperties = partiallyApply( prepareData, properties );
composedFunction = flow(
  adjustData,
  prepareDataWithProperties,
  doSomething
);
composedFunction( data );

Now the partially applied version can be used in our pipe because its first argument is already saved. This is like knowing what kind of tea to make already. We go from needing two pieces of information to only needing one.

There are many ways to actually use partial application in practice. In JavaScript we can use bind and the Lodash flow methods to achieve the above example. But it’s a big topic and there’s a lot of other options available. I’m still learning about them myself.

Avoiding Tightly-Coupled REST APIs

As you might have read, WordPress is getting a powerful REST API. For a few years now I’ve been writing endpoints for the WordPress.com version of this API, as well as few REST APIs in other languages (Ruby, JavaScript). As I’m also a big fan of unit testing, I’d like to share a pattern that I’ve learned from trial and error.

To make REST API endpoints more testable, as well as more useful, write the logic in a separate library.

Let’s say I have an endpoint that updates a menu item on a restaurant web site. In WordPress I have the menu item set up as a custom post type (you don’t need to grok WordPress to benefit from this pattern, but I thought I’d mention it).

The basic operation of the endpoint is this:

  1. Take the post ID and the new field values from the data submitted to the endpoint.
  2. Find the matching post in the database.
  3. Verify user permissions to update that post.
  4. Calculate the menu item’s total price based on current tax rate and ingredients.
  5. Update the post’s data.
  6. Return the newly updated post data to the client.

I’d also like to write unit tests for each of the above operations.

Now, the first step in writing my endpoint would be to map the HTTP route to the route handler. That is, I need to map the route PUT /menu/item to a function which kicks off the process above and eventually returns the result to the client.

I could write all of the above logic right in that function. There’s even a few pieces which could be their own separate functions. But this way may not be ideal, as I’ll explain.

The route handler, wherever it might be defined, is aware that it’s part of a REST API. The exact details of that vary depending on which language, architecture, etc, that you are using. But generally the initial route handler is not just doing the work of the steps above, but also interacting in some way with the API. Perhaps it’s checking details of the path, query string, user token, or it’s sending back an HTTP-specific response.

Here’s where we can get into trouble. Let’s say later on we decide to write another REST API endpoint that accepts a new tax rate and updates all the menu items. It needs to do the following:

  1. Take the new tax rate from the submitted data.
  2. Iterate over each menu item post.
  3. Validate user permissions to update each post.
  4. Calculate each menu item’s new total price.
  5. Update each post.

Notice that steps 3, 4, and 5 are the same as the previous endpoint we made. Rather than re-write them here, let’s keep things DRY and just call the other endpoint; thus our new update-the-tax-rate endpoint looks more like this:

  1. Take the new tax rate from the submitted data.
  2. Iterate over each menu item post.
  3. Call the update-menu-item endpoint.

Ah… but in the WordPress REST API, as well as many other API frameworks in different languages, you can’t call an API endpoint from within another API endpoint. And it might be said that, even if you could, doing so adds unnecessary overhead.

In this case, if the process of finding, validating, and updating a post was all one function call in a library, maybe update_menu_item(), then this endpoint becomes so much easier. Both the update-menu-item endpoint and the update-the-tax-rate endpoint can just call update_menu_item().

Even better, so can our unit tests! While it’s a good idea to write tests for an API endpoint that actually call the API, doing so can sometimes be problematic. Unit tests usually try to test one specific behavior, separating that behavior from the rest of the system. If you test a REST API, the system being tested is on a separate server (even if the server is local to your computer), and is thus much more complicated to control. But if the API’s route handler simply calls a function in a library, we can just test that function, providing mock data and checking the results, all without even considering HTTP.

Certainly this isn’t a rule, as most rules in programming tend not to fit real-life situations. It’s just a pattern I’ve decided works well for me. I suggest you examine your own code to see if it might work for you too!

get_deep_value in PHP

Ever have an array in PHP that looks like this?

$data = [ 'one' => [ 'two' => [ 'four' => 'bar', 'three' => 'foo' ] ] ];

It can be annoying to pull deep values from such an array because if you access an array key that doesn’t exist, PHP will throw an exception.

do_something_with( $data['one']['two']['six'] ); // throws an exception!

You could check each level of the data using array_key_exists() like this:

if ( array_key_exists( 'one', $data )  && array_key_exists( 'two', $data['one'] ) && array_key_exists( 'three', $data['one']['two'] ) ) {
  do_something_with( $data['one']['two']['three'] );
}

But that would take forever. PHP’s isset() method fortunately makes this somewhat more clean:

if ( isset( $data['one']['two']['three'] ) ) {
  do_something_with( $data['one']['two']['three'] ); 
}

But that’s still a lot to type, especially if you have to do it several times. In JavaScript-land we have some nice patterns around this sort of thing. Underscores and Lodash have the _.get() method, so I’ve re-created that in PHP here as get_deep_value().

The first argument is the array you want to search, and the second argument is an array of keys, one per level.

Now you can just write:

if ( $value = Get_Deep_Value::get_deep_value( $data, ['one', 'two', 'three'] ) {
  do_something_with( $value ); 
}

Or maybe even, if your function is ok with null values, just simply:

do_something_with( Get_Deep_Value::get_deep_value( $data, ['one', 'two', 'three'] ) );

I’m just posting the gist here for now because I’m not quite sure how to release this as a general module in the PHP ecosystem yet. If you have any helpful suggestions, please add them in the comments!

The “why” of writing tests

Unit testing is part of my normal development flow now. For the most part, I try to write my code in a “TDD” or “BDD” manner, meaning: I write my tests first, explaining how I want my public methods to behave, then follow it up with the actual code to make the tests pass.

Years ago I was convinced that this was the best way to do things, and my own experience has backed up that decision. But, as is the case with many decisions from the past, I’ve started to forget exactly why.

I recently found myself in the role of teaching some other developers my testing practices. When I began, I thought the hardest part would be to explain the technical details: the ins an outs of test runners, mocking, and assertions. While those things are certainly important, as it happened the hardest part was trying to explain why we were writing unit tests at all.

In this particular project I’m writing unit tests for code that I don’t fully understand. As a result, the process of writing the tests (or reading tests written by others) is more about learning. This was my first realization:

Writing tests forces the coder to have a complete understanding of the code they’re testing.

Ultimately, tests have to do with behavior. Given certain inputs, we expect certain output. If I turn the key, I expect the engine to start. That’s a unit test, of a sort: it’s testing the “output” of the engine starting with the “input” of turning the key. But that may also be considered an integration test, because the engine is rather complex, and if my test were to fail, it might be due to any number of factors.

So let’s go one level deeper: If I turn the key, I expect the spark plugs to ignite. That’s another type of unit test, and a much smaller unit. I think that this is the level I usually write tests for. But of course there’s still smaller units within that, all the way down to the physical materials of copper and steel. At some point, you have to trust that some units are just going to work, and write your tests one level above that. This led me to my second realization:

All tests are unit tests, just for differently-sized units. The best unit tests will not only tell you when something is broken, they will also tell you why.

My favorite testing mantra is “red, green, refactor”. If you write your test first and try to run it, it will fail (“red”). Next you write the most basic version of the code to make the test pass (“green”). Finally you can safely refactor the code to be cleaner, as long as its public API remains the same, because the tests will continue to pass (“refactor”). If you want to change the behavior, you start again at “red” by changing the test first. This led me to my third realization:

Testing forces you to consider exactly what you want the behavior to be, and therefore acts as living documentation for how to use your code.

I’m sure there are as many reasons to test code as there is code to be tested, but I’m starting to appreciate my own testing experiences more. I think they’ve made me a better developer, and I hope I’m able to communicate that to others.

Featured image: By Andy Dingley (scanner) (Scan from (1911) Mechanical Transport, HMSO) [Public domain], via Wikimedia Commons

Using React stateless components for quick prototyping

When you’re building a new app using React it’s nice to start laying out all the components you’ll want to use inside the components that you’re building. Unfortunately this can be a little slow because for each new component you have to:

  1. Create a new file
  2. Import that file
  3. Include React in the file
  4. Export a Component from the file
  5. Add a simple render method to the Component

Step 5 is really the core of what you want to do here, and I realized today that I can quickly skip the other four steps by just using React Stateless Components, which are just functions. When using ES2015 fat-arrow syntax, they can be one-liners!

Using these you can mock up your Component layout very quickly, then move them over to your new Component files one at a time as you are ready. And you may even have some of your render methods there for you!

Re-ordering Objects in a List

In an old version of an app I wrote, I had a list of objects which were in a specific order. I wrote the app to operate using a REST API, so when you changed one of the objects, the client sent a PUT request to the server with the new data for that object and the change was made.

This worked very well for everything except then I got into a bind trying to figure out how to change the order of the objects in the list.

The way I did it back then was… not the greatest. I assigned a property to each object called order which was the index of that object in the list. When the user moves an object, the client sends a PUT request to the server telling it the new value of order. So if you moved the object at position 5 to position 2, my app would ask to change that object to have an order of 2.

Sounds easy enough. But wait, there’s already an object with the order of 2. What do we do with that? So we have to move it down to 3. In fact, we have to move all of the objects below that down one index, up until we get to the old index (because we don’t want to move the object with the order of 6).

That’s a lot of work (and a lot of changed models), especially for long lists. It’s also an entirely different operation if you’re moving objects down the list versus moving objects up the list (you have to update the objects in the reverse direction). And to make it extra-hard, my code needed to perform this complex operation twice: once on the client before we sent the data (since the data sent to the server was mirrored on the client), and once on the server.

Recently I was going over this code again and I saw what amounted to a pretty big mess that I was sure could be done differently. I started thinking of a deck of playing cards, and how easy it is to just move one card from one place in the deck to another. How simple! Could I achieve the same simplicity here?

My first realization was that there’s already a type of object in programming languages that handles re-ordering very efficiently, just like a deck of cards: the humble Array.

I was so fixated before on the idea that each object had to have an order property, that I missed what may otherwise have been obvious. When I re-order something, I’m really re-ordering the whole list, which is just like re-ordering an Array.

By removing the order property from each object on the client, I no longer need to update each object when the order changes; the objects simply have different places in the Array. Then I just send the whole new array back to the server (actually just the IDs of the objects) and the server updates its version of the list as well.

I’m sure this is not a new idea, but it was new to me, and it’s saved me a lot of work. It just goes to show how we evolve as developers over the years and how it’s probably always worth going back over old code.