API documentation as code with Swagger, AsciiDoc, Cucumber and Structurizr

design jun 21, 2016

With the growing love of API at zooplus, more our apps & services are provided with a RESTful API. We love RESTful API because it’s simple and flexible.

To keep the simplicity and flexibility, we need to be proactive in providing accurate information about our API to the users, partners and business. But traditional documentation tends to be problematic to maintain and often turns out to be too easily out-dated.

So what is our solution?

As a developer, code is our only best friend. Whatever can best describe our code is probably the best documentation we can get. Thus the strike for a Living Documentation - something that we can actually write with code.

For one of our recent APIs, we’ve produced following documentation all generated by code:

API specification
API user reference documentation
Use case / test scenario report
Architecture diagrams

We target to cover a wide range of audience with them: API users, business stakeholders and techies @ zooplus. And those documentation really look good, what’s the matter to produce something that nobody is willing to read anyway?

Now let me work you through our journey to create those.

Kick-off with Swagger specification

As many people do, we start by specifying the basic skeleton of our API using Swagger. Then we code a functional prototype right away with Spring Boot, add Swagger Annotations and expose the refreshed spec via /api-docs endpoint using Springfox. After that we simply put our API into iterations and keep making changes to the spec.

The good thing of the spec-code-spec iteration approach is we can get an always-up-to-date spec of our API. What is still missing however, is the possibility to have it available offline and have it versioned.

To address this, we wrote a small test against /api-docs, save the response into a JSON file, package and deploy it as a separate artifact using maven-assembly-plugin:

Note: spring-boot 1.4.x is used in this example.

@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
@ActiveProfiles("dev")
public class ApiDocumentationTest {

    @Autowired
    WebApplicationContext appContext;
    MockMvc mockMvc;

    @Before
    void setup() {
        mockMvc = MockMvcBuilders
            .webAppContextSetup(appContext).build();
    }

    @Test
    void writeApiDocs() throws Exception {
        // document swagger spec
        mockMvc.perform(get("/api-docs")
                .accept(MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andDo(documentSwaggerJson("target/generated-docs"));
    }

    ResultHandler documentSwaggerJson(String outputDir) {
        return result -> Files.write(
            Paths.get(outputDir, "swagger.json"),
            result.getResponse().getContentAsByteArray());
    }
}

At this point, we are able to get an offline spec of our API after each build and keep it tracked by artifact version. Now we are free to share it with the counter-parties, at any time we want and with any version we want, even before we roll out the API for instance.

Tip: With Artifactory, we can also directly link to the latest version of your artifact, so the link never get out-dated.

The Swagger spec is nice, with Springfox you can even package a Swagger UI along with your API. But none of them is good to provide a easy-to-read, rich-content reference doc to your API.

For that we have added an AsciiDoc to our API , combining the specification with hand written documentation, all with a nice looking generated layout, thanks to Robert Winkler’s inspiring article.

To put it with our sauce, we first enhanced our test mentioned before with additional steps:

Note: For this to work you may need to add spring-restdocs, swagger2markup and swagger2markup-spring-restdocs-ext as test dependencies to your project.

@Rule
JUnitRestDocumentation restDocumentation =
    new JUnitRestDocumentation("target/generated-docs/snippets");

@Before
void setup() {
    mockMvc = MockMvcBuilders.webAppContextSetup(appContext)
        .apply(documentationConfiguration(restDocumentation))
        .build();
}

@Test
void writeApiDocs() throws Exception {
    // document operation example snippets
    mockMvc.perform(post("/orders/{id}", 123L)
            .accept(MediaType.APPLICATION_JSON)
            .contentType(MediaType.APPLICATION_JSON)
            .content(readFile("api_docs_sample_order.json")))
        .andExpect(status().isCreated())
        .andDo(documentOperation("createOrder"));

    // document swagger spec and reference doc with snippets
    mockMvc.perform(get("/api-docs")
            .accept(MediaType.APPLICATION_JSON))
        .andExpect(status().isOk())
        .andDo(documentSwaggerJson("target/generated-docs"))
        .andDo(documentSwaggerAsciiDocs("target/generated-docs/snippets"));
}

The test generates an AsciiDoc version of the API spec breaking down into 3 snippets under target/generated-docs folder: overview.adoc, paths.adoc and definitions.adoc. It also includes a sample request/response for each tested operation (for this you need to make sure the operationId is defined for all API operations).

Now we can add hand-written content as we want (e.g. src/api-docs/limitations.adoc) and include code snippets as well:

== Limitations
=== First limitation of our API
Some text here...
Include code snippets:
----
include::../../src/main/java/com/zooplus/SomeCode.java[indent=0]
----

To put everything together, we still need to create an src/api-docs/index.adoc snippet file:

include::snippets/overview.adoc[]
include::snippets/paths.adoc[]
include::snippets/definitions.adoc[]
include::limitations.adoc[]

At this point, we have all our hand-written snippets under src/api-docs and all generated snippets under target/generated-docs. To prepare for the final rendering, we need to put all the snippets in one place. In our case, we use maven-resources-plugin to copy all contents from src/api-docs to target/generated-docs before rendering.

Finally, we need AsciiDoctor to render the whole thing into HTML. And in our case we used asciidoctor-maven-plugin for that.

The final result is something like this:

The only thing left is, again, package target/api-docs and deploy it as a separate artifact using maven-assembly-plugin. Then we are free to share the nice reference doc with the world.

Tip: With Artifactory, we can actually serve HTML from an archive if jar/zip content browsing is enabled.

Add more context with cucumber reports

Cucumber is widely used for writing use cases and business scenarios for testing. Sometimes it’s just a lot easier to show an example instead of writing thousands of words to explain a feature. Since we already have those Cucumber scenarios anyway, why don’t use them in our documentation as well?

With cucumber-reporting we can easily generate nice looking reports out of cucumber scenarios and add it as part of our build process. It also provide a maven plugin that we use for generating reports like this:

But that’s not done yet. Once again we package and deploy the generated reports as a separate artifact using maven-assembly-plugin. So that we can actually link to it from our other documentation.

One use case for us is, we want to explain a limitation that our API currently has and we think it’d be more clear to include a living example scenario. So we put a deep link to the cucumber report in the reference doc (i.e. limitations.adoc):

=== Limitation of our API
Some explanation here...
Please refer to
http://link/to/artifact/artifact-{version}-cucumber-reports.zip!/feature_1.html[Feature scenario 1] for more information.

The {version} placeholder will take the actual value from the build. So that we keep the content of our reference doc consistent to the cucumber reports.

Final touch with Structurizr diagrams

Till now, we got the specification, reference docs and business scenarios done so that our users and business stakeholders can get up-to-date information about our API. A final touch is still needed to make our techie readers happy: some shiny architecture diagrams.

From the workshop we’ve got from Simon Brown, we get to know this nice service Structurizr, gracefully developed by Simon, that allows us to code and generate C4 diagrams for our API.

Setup it up is really simple. Everyone could do it in a matter of minutes by following the tutorials. To put it in our case, we created another test to manually setup the system and containers involved in our product, then used the component finder feature to automatically discover the components we have:

Note: For this to work you may need to add structurizr-core and structurizr-spring as test dependencies to your project.

@Test
void createContainerDiagram() {
    // create your workspace
    Workspace workspace =
        new Workspace("Order Mgt. APIs", "C4 models...");
    Model model = workspace.getModel();

    // add software systems
    SoftwareSystem mgtApis =
        model.addSoftwareSystem("Order Mgt. APIs", "The system...");

    ...

    // add containers
    Container apiGateway =
        mgtApis.addContainer("Order API Gateway", "...", "Nginx");
    Container validationApi =
        mgtApis.addContainer("Order Validation API", "...", "Java");
    Container creationApi =
        mgtApis.addContainer("Order Creation API ", "...", "Java");

    ...

    // link systems <-> containers, containers <-> containers
    model.getSoftwareSystems()
        .forEach(s -> s.uses(apiGateway, "uses", "http"));
    mgtApis.getContainers()
        .forEach(c -> apiGateway.uses(c, "reverse proxy", "http"));

    ...

    // customize diagram styles
    ViewSet viewSet = workspace.getViews();
    viewSet.getConfiguration().getStyles()
        .addElementStyle(Tags.CONTAINER)
            .background("#9fc5e8").color("#ffffff")
        .addElementStyle(database.name)
            .background("#e06666").shape(Shape.Cylinder);

    ...

    // create diagram
    ContainerView containerView =
        viewSet.createContainerView(mgtApis, "APIs overview");
    containerView.addAllSoftwareSystems();
    containerView.addAllContainers();

    structurizrClient.mergeWorkspace(WORKSPACE_ID, workspace);
}

@Test
void createComponentDiagram() {
    // auto-discover spring @Controller, @Service, @Repository etc.
    ComponentFinder finder = new ComponentFinder(
        creationApi, "com.zooplus",
        new SpringComponentFinderStrategy());
    finder.findComponents()

    ...
}

After the first run, we could arrange the elements of our diagrams directly on structurizr.com using the provided UI and save the layout of our diagrams once. Then we made sure the test is included in our build pipeline so the diagrams will get updated at every build.

The outcome looks like this:

Now we can include those diagrams into our documentation and never have to worry that they get out-dated again. Awesome isn’t it?

With all the different techniques we’ve addressed in this article, we are no more afraid to create documentation for our products.

Vive the living documentation!