Apple ran hundreds of tests a week on software it did not control.

I started at the end. Before drawing anything I worked backwards from what a finished test had to produce, until the system satisfied both the business and the person running it.

There was no recruitment and no screener. There was also no research team, so this was guerrilla work: whiteboards, notes, and competitor applications pulled apart one at a time.

Three decisions had a real alternative on the table. Recording what was rejected matters more than recording what shipped. Without the alternative on record you cannot tell a decision from a default.

From requirements to structure

Requirements were written one on one with the marketing team: whiteboarding, note-taking, competitor review, workshops. The sitemap was built alongside the requirements document rather than after it, so the scale of the system and the way a user would move through it were understood together.

The sitemap as it was actually drawn. Three parallel tracks, proof of concepts, proposals and projects, each running the same chain: experiments, setup, validation, then QA, staging and production. The dashed boxes are gates, not pages.

Four versions of the dashboard, drawn at the time and rebuilt here from those files. The organising idea moves each time: name the sections, then lead with the result, then organise by store, then compress the states. The calendar came out of this run, not before it.

The navigation moves with it, which is why these four do not match the screens further down. Dashboard, Projects, Archives becomes Stores and Experiment Request, and only then settles into the seven sections the finished platform shipped with. It took four rounds to work out that the store tree was the organising structure and everything else hung off it.

Before any screen existed, the shape of the work was already fixed by what a test is. Five steps, in this order, every time. The platform’s job was to make each one obvious and none of them leave the building.

Navigation and task flow

Two factors drove the navigation: the type of test, and where it runs. The system also had to separate tools and roles, because not every user had access to every part of it.

The flows above show one task in order. The map below covers every screen in the platform, what it holds, and where it goes next.

The inbound list is derived by inverting the graph rather than written down, so a screen nothing reaches shows up as one.

Structure first, blocked out coarse enough that nobody could mistake it for a design. These are redrawn: the originals were not preserved.

The three questions a tester has before running a test.

Every screen below is a live recreation, not a screenshot.

Tested while designing

Because I sat with the team that would use the tool, testing was not a phase at the end. I could build a prototype, put it in front of a tester the same day, and fold what came back into the next version. The design moved as we worked rather than waiting on a formal round.

No formal test report survives from the project, so what is shown here is the design those sessions produced rather than the sessions themselves.

What this project taught me.

• Working backwards from the finished outcome was faster than designing forwards from a feature list.
• Being embedded with the users removed most of the research overhead a team of one could not otherwise afford.
• The security problem was solved by an org decision, routing through Apple Directory, not by an interface. Knowing which problems are not design problems mattered more than any screen.
• I left at detailed visual design, so I did not see the built product ship.