Avatar

Blog

  • Published on
    When you're building .NET APIs you're comfortably surrounded by tooling for generating data and operation contract documentation automatically for you. The combination of SwaggerGen and SwaggerUI can automatically present consumers of your API with everything they need to understand the interface and to scaffold clients/DTOs - all based on the generated OpenAPI specification file, without much work as the developer. However when you're building something that isn't an "API" in the sense of "HTTP Request/Response", such as building serverless functions that process incoming messages you lose an awful lot of that tooling. In some eventing systems (such as Kafka) you have a schema registry, so you can use that to enforce validity and versioning of message data contracts for producers and consumers of that data (for example with an Avro schema). For simpler setups with no schema registry, it's still nice to have automatically generated documentation and schemas based on your source code. The reason I suggest generating your documentation for source is it ensures correctness - there's only one thing worse than missing documentation and that's wrong documentation. Also assuming you're versioning your software you'll have corresponding versioned documentation that sits alongside. The two goals of my approach are:
    • Produce JSON schema files for machine consumption
    • Produce Markdown files for human consumption

    JSON Schema

    Using JSON schemas are a great way to share data contracts in a language agnostic way, so consumers in any language can scaffold types from your schemas. They can also be used to validate incoming JSON before de-serializing it. Not only that but they plug into the OpenAPI specification, so you can reference them in your spec file (for example if you're exposing the data contract via an API Gateway endpoint). You can easily generate a JSON schema for a C# type using NJsonSchema, for example:
    
    var typesToGenerate = GetTypesToGenerate(); // todo: implement creating the list of types, e.g. using typeof(MyDotnetModels.SomeType).Assembly.GetTypes()
    
    if (!Directory.Exists("./generated"))
    {
        Directory.CreateDirectory("./generated");
    }
    
    foreach (var type in typesToGenerate)
    {
        var schema = JsonSchema.FromType(type);
    
        // add required to all non-nullable props
        foreach (var propertyInfo in type.GetProperties(BindingFlags.Public | BindingFlags.Instance))
        {
            var isNullable = Nullable.GetUnderlyingType(propertyInfo.PropertyType) != null;
            if (!isNullable)
            {
                schema.RequiredProperties.Add(propertyInfo.Name);
            }
        }
        
        var schemaJson = schema.ToJson();
        
        File.WriteAllText($"./generated/{type.Name}.schema.json", schemaJson);
    }
    
    Build that code snippet into a console app (e.g. called JsonSchemaGen) and you can now execute it whenever your build your source code for deployment and it will generate JSON schema files in the bin/generated folder.

    Markdown

    Now that we have JSON schemas, it's easy to generate markdown files using @adobe/jsonschema2md. Simply pass the location of schema files with any configurations of your choice, e.g.
    
    npx -y @adobe/jsonschema2md -h false \
    	-d [path_to_schema_files] \
    	-o markdown -x -
    
    That will generated a README.md and md files for each schema, with human readable content describing your data contracts.

    Using a makfile to combine it all

    This part is optional, but it's nice to have the commands necessary to perform the above steps checked in to source control and to use the power of "make" to run the necessary steps only when things change, e.g.
    
    .PHONY: clean build
    
    schemas: [relative_path_to_dotnet_source]/MyDotnetModels
    	cd JsonSchemaGen && \
    	dotnet build -c Release && \
    	cd bin/Release/net8.0 && ./JsonSchemaGen && \
    	mkdir -p ../../../../schemas && mv generated/* ../../../../schemas/
    	
    markdown: schemas
    	npx -y @adobe/jsonschema2md -h false \
    	-d ./schemas \
    	-o markdown -x -
    	
    build: markdown
    	
    clean:
    	rm -rf schemas markdown
    
    Summary of the steps:
    • schemas - depends on the dotnet models source code directory, so this runs whenever any file in that directory changes
      • Build the console app (including a reference to models) in release mode
      • Execute the built console app
      • Moves the generated files in another folder to make them easier to find
    • markdown - depends on the schemas, so this runs whenever the schemas change
      • Use npx to execute the @adobe/jsonschema2md and output to a directory called 'markdown'
    You can now incorporate "make build" of your documentation makefile into your CI process and store the "markdown" and "schemas" directories as build artifacts, alongside your system build artifacts. Now they are ready to be shipped with the system when it's released (e.g. put in a storage account, or hosted as a static website - for example).
  • Published on
    When building CI/CD pipelines it is often the case that you'd like to "tokenize" a configuration file, so that the values in the file can be calculated during the build/release execution process. This can be for all kinds of reasons, such as ingesting environment variables or by basing values on the outputs of other CLI tools, such as terraform (e.g. reading IDs of recently created infrastructure). Rather than separating the key/value "tokens" from the "token calculation" logic, I wanted a way to embed Bash scripts directly into my JSON file and then have interpret the values into an output file. For example, the JSON file might look like:
    
    [
        {
            "Key": "verbatim",
            "Value": "just a string"
        },
        {
            "Key": "environment-variable",
            "Value": "$HOME"
        },
        {
            "Key": "shell-script",
            "Value": "$(dotnet --version)"
        }
    ]
    
    With the invokable tokens defined, the below script can be run against the JSON file in order to parse and execute the tokens:
    
    #!/bin/bash
    
    jsonFile=$1
    tempFile=$2
    
    echo "[" > "$tempFile"
    
    itemCount=$(jq '. | length' $jsonFile)
    currentIndex=0
    
    # Read and process the JSON file line by line.
    jq -c '.[]' $jsonFile | while read -r obj; do
        key=$(echo "$obj" | jq -r '.Key')
        value=$(echo "$obj" | jq -r '.Value')
    
        # Check if value needs command execution
        if [[ "$value" == \$\(* ]]; then
            # Remove $() for command execution
            command=$(echo "$value" | sed 's/^\$\((.*)\)$/\1/')
            newValue=$(eval $command)
        elif [[ "$value" == \$* ]]; then
            # It's an environment variable
            varName=${value:1} # Remove leading $
            newValue=${!varName}
        else
            # Plain text, no change needed
            newValue="$value"
        fi
    
        # Update the JSON object with the new value
        updatedObj=$(echo "$obj" | jq --arg newValue "$newValue" '.Value = $newValue')
    
        # Append the updated object to the temp file
        echo "$updatedObj" >> "$tempFile"
    
        # Add a comma except for the last item
        ((currentIndex++))
        if [[ $currentIndex -lt $itemCount ]]; then
            echo ',' >> "$tempFile"
        fi
    done
    
    echo "]" >> "$tempFile"
    
    Running the command looks like
    ./json_exec.sh example.json example.out.json
    And the output then looks as follows:
    
    [
    {
      "Key": "verbatim",
      "Value": "just a string"
    }
    ,
    {
      "Key": "environment-variable",
      "Value": "/home/craig"
    }
    ,
    {
      "Key": "shell-script",
      "Value": "8.0.202"
    }
    ]
    
  • Published on
    Imagine you want to create a very generic SpecFlow step definition that can be used to verify that a certain HttpMessageRequest was sent by your system that uses HttpClient. You want to check your system calls the expected endpoint, with the expected HTTP method and that the body data is as expected. The gherkin syntax for the method might be something like:
    Then the system should call 'POST' on the 3rd party 'hello-world' endpoint, with the below data
      | myBodyParam1 | myBodyParam2 |
      | Hello        | World        |
    
    C# being a strongly typed language, it's actually not that straightforward to make a robust comparison of the JSON that was sent in a request, with a Table that is supplied to SpecFlow. However, I did manage to come up with such a way, which is documented below.
    
    [Then(@"the system should call '(.*)' on the 3rd party '(.*)' endpoint, with the below data")]
        public void ThenTheSystemShouldCallOnThe3rdPartyEndpointWithTheBelowData(HttpMethod httpMethod, string endpointName,
            Table table)
        {
            var expectedRequest = table.CreateDynamicInstance();
    
            _mock.VerifyRequest(httpMethod, 
              async message => message.RequestUri!.AbsoluteUri.EndsWith(endpointName) &&
                await FluentVerifier.VerifyFluentAssertion(async () =>
                  JsonConvert.DeserializeAnonymousType(
                    await message.Content!.ReadAsStringAsync(),
                    expectedRequest).Should().BeEquivalentTo(expectedRequest)));
        }
    
    There's several parts to the magic, in order:
    1. `table.CreateDynamicInstance` - this extension comes from the SpecFlow.Assist.Dynamic NuGet package, which allows you to create an anonymous type instance from a SpecFlow table.
    2. `_mock.VerifyRequest` - this extension comes from the Moq.Contrib.HttpClient, which isn't strictly necessary but is a nice way to manage your HttpClient's mocked message handler and make assertions on it.
    3. `await FluentVerifier.VerifyFluentAssertion` - uses this trick for making FluentAssertions inside of a Moq Verify call (so you can use equivalency checks rather than equality).
    4. `JsonConvert.DeserializeAnonymousType` - allows you to deserialize JSON to an anonymous type based on a donor "shape" (which we get from the "expected" anonymous type)
  • Published on
    Cypress is a great choice of tool for writing e2e UI automation tests for web applications. One of things you'll invariably want to do when writing tests is stub the dependencies of your system, so that you can isolate the SUT and allow for performant test runs. If you're testing a purely client side code-base, like React, then the built-in cy.intercept might do the job. This will intercept calls made from the browser to downstream APIs and allows you to stub those calls before it leaves the client. However for a Next.js application that includes server side data fetching (e.g. during SSR) or where you have implemented Next.js APIs (e.g. for "backend for frontend" / "edge" APIs) that you want to include as part of the "system under test", you need another option. The way the Cypress Network Requests documentation reads, it seems like the only choices are mocking in the browser using cy.intercept, or spinning up your entire dependency tree - but there is a 3rd option - mocking on the server. Mocking server side calls isn't a new paradigm if you're used to automation testing C# code or have used other UI testing frameworks so I won't go into major detail on the topic, but I wanted to write this article particularly for Cypress as the way you interact with the external mocks is different in Cypress. To mock a downstream API, you can spin up stub servers that allow you to interact with them remotely such as WireMock or in this case "mockserver". What you need to achieve will comprise of these steps:
    1. Before the test run - spin up the mock server on the same address/port as the "real" server would run (actually you can change configs too by using a custom environment, but to keep it simple let's just use the default dev setup)
    2. Before the test run - spin up the system under test
    3. Before an "act" (i.e. during "arrange") you want to setup specific stubs on the mock server for your test to use
    4. During an "assert" you might want to verify how the mock server was called
    5. At the end of a test run, stop the mock server to free up the port
    In order to orchestrate spinning up the mock server and the SUT, I'd recommend scripting the test execution - which you can read more about here - below shows an example script to achieve this: test-runner.mjs
    
    #!/usr/bin/env node
    import { subProcess, subProcessSync } from 'subspawn';
    import waitOn from 'wait-on';
    const cwd = process.cwd();
    
    // automatically start the mock
    subProcess('test-runner', 'npm run start-mock', true);
    await waitOn({ resources: ['tcp:localhost:5287'], log: true }, undefined);
    
    // start the SUT
    process.chdir('../../src/YourApplicationRoot');
    subProcess('test-runner', 'make run', true);
    await waitOn({ resources: ['http://localhost:3000'] }, undefined);
    
    // run the tests
    process.chdir(cwd);
    subProcessSync("npm run cy:run", true);
    
    // automatically stop the mock
    subProcess('test-runner', 'npm run stop-mock', true);
    
    process.exit(0);
    
    That suits full test runs and CI builds, but if you're just running one test at a time from your IDE you might want to manually start and stop the mock server from the command line, which you can do by running the "start-mock" and "stop-mock" scripts from the CLI, hence why they have been split out. start-mock.js
    
    const mockServer = require('mockserver-node');
    
    mockServer.start_mockserver({ serverPort: 5287, verbose: true })
      .then(() => {
        console.log('Mock server started on port 5287');
      })
      .catch(err => {
        console.error('Failed to start mock server:', err);
      });
    
    stop-mock.js
    
    const mockServer = require('mockserver-node');
    
    mockServer.stop_mockserver({ serverPort: 5287 })
      .then(() => {
        console.log('Mock server stopped');
      })
      .catch(err => {
        console.error('Failed to stop mock server:', err);
      });
    
    package.json:
    
    "scripts": {
        "start-mock": "node start-mock.js",
        "stop-mock": "node stop-mock.js",
        "test": "node test-runner.mjs",
        "cy:run": "cypress run"
      },
    
    With the mock server and SUT running you can now interact with them during your test run, however in Cypress the way to achieve this is using custom tasks. Below shows an example task file that allows you to create and verify mocks against the mock-server: mockServerTasks.js
    
    const { mockServerClient } = require('mockserver-client');
    
    const verifyRequest = async ({ method, path, body, times = 1 }) => {
      try {
        await mockServerClient('localhost', 5287).verify({
          method: method,
          path: path,
          body: {
            type: 'JSON',
            json: JSON.stringify(body),
            matchType: 'STRICT'
          }
        }, times);
        return { verified: true };
      } catch (error) {
        console.error('Verification failed:', error);
        return { verified: false, error: error.message };
      }
    }
    
    const setupResponse = ({ path, body, statusCode }) => {
      return mockServerClient('localhost', 5287).mockSimpleResponse(path, body, statusCode);
    };
    
    module.exports = { verifyRequest, setupResponse };
    
    This can then be imported into your Cypress config:
    
    const { defineConfig } = require("cypress");
    const { verifyRequest, setupResponse } = require('./cypress/tasks/mockServerTasks');
    
    module.exports = defineConfig({
      e2e: {
        setupNodeEvents(on, config) {
          on('task', { verifyRequest, setupResponse })
        },
        baseUrl: 'http://localhost:3000'
      },
    });
    
    Finally, with the custom tasks registered with Cypress, you can use the mock server in your tests, e.g.:
    
    it('correctly calls downstream API', () => {
        // setup API mock
        cy.task('setupResponse', { path: '/example', body: { example: 'response' }, statusCode: 200 }).should('not.be.undefined');
    
        // submit the form (custom command that triggers the behaviour we are testing)
        cy.SubmitExampleForm();
        
        cy.contains('Example form submitted successfully!').should('be.visible');
    
        // verify API call
        cy.task('verifyRequest', {
          method: 'POST',
          path: '/example',
          body: {
             example: 'request'
          },
          times: 1
        }).should((result) => {
          expect(result.verified, result.error).to.be.true;
        });
      });
    
  • Published on
    I like to work in pictures - and I use diagrams A LOT as a way to express the problem or solution in a digestible format. A picture is worth a thousand words, after all! In the early stages of understanding, I find it's best to stay in a diagramming tool like "draw.io" - you can quickly throw things onto a page, often in real-time while collaborating on a screen share and let the diagram drive the narrative of the conversation. The outputs of these diagrams can be easily shared for all to see and can quickly build up a dossier of useful documentation when it comes to explaining or coding the solution down the line. I tend not to worry so much at this stage about following any particular "style" of diagram, with draw.io supporting everything from swim lanes, flow charts, deployment diagrams, component diagrams and more, I usually go for whatever combination is the most useful or informative for the situation. The important measure for a diagram is that everyone, who needs to, can understand it. As you get closer to an agreed architecture, there comes a point where it's worth moving to a "model" over a "diagram" (by that I mean not just pictures, but written syntax that can be output as diagrams - i.e. "documentation as code"). What springs to mind here is PlantUML (or Mermaid diagrams) which allow you to define a plethora of diagram types using written syntax. The main benefit of this in my experience is change control, as written syntax plays very nicely with source control systems, such as Git so you can request peer reviews of documentation changes, keep it aligned with code changes and can see the history of changes with commit messages as to why things changed. As a "standard" set of diagrams to get started, I'd recommend the "C4 Model" - this will give you a good repeatable basis for capturing the system architecture at various levels of abstraction. I don't really recommend going beyond components and even then, I'd use that level of detail sparingly. However, both PlantUML and Mermaid support multiple other diagram types so it's worth having a dig through to see what you find useful. I especially like to create sequence diagrams as text, as I find that easier than doing it graphically. You can even graphically represent JSON or YAML data, create pie charts and more! I tend to categorise documentation into "specific" or "overarching" - "specific" documentation you'd find in the Git repo of the specific system it relates to, typically in a "/docs" folder alongside "/src" and here you will find the software and solution level documentation focused on the internal workings of this system, with perhaps minimal references to the external systems it interacts with (e.g. at the "context" level). Some good examples would be the lower level C4 diagrams, sequence or flow diagrams etc. Sometimes though, documentation focuses on the interaction between systems rather than a specific system, or it's documenting the business processes using mind maps etc. and isn't "about" any particular API. In that case to me that's "overarching" documentation and I'd have a dedicated repo for that genre of documentation. Unitl recently, I was using an open source tool called "C4Builder" to manage my documentation projects. It allows you to setup a structure of "md" (markdown) and "puml" (plantUML) files and then generate them into a HTML/Image output, which can be hosted directly in the Git repository. There are plugins for VS Code and JetBrains Rider that allow you to code Markdown/PlantUML with previews, while you are working on the documentation, which c4builder can then "compile". However, this tool seems to no longer be maintained so it doesn't support the latest version of PlantUML which limits what diagrams and syntax are available to you - I have created a Docker image version that monkey patches in the latest (at the time of writing) PlantUML so you can at least continue to build .c4builder projects, but I've now discovered a new way to manage my documenation: JetBrains Writerside. Writerside is still in EAP and has just announced support for PlantUML, it already supports Mermaid diagrams and has a tonne of other useful features for creating and publishing technical documentation, such as supporting OpenAPI specs and markup/markdown elements so it's well worth a look! To sum up, I recommend creating diagrams throughout the software development process, including during requirements gathering to visualise and get a collective understanding of the problem and agreement on proposed solution(s). As you move towards starting to write code, formalise the necessary (not everything) documentation into a syntactic model that is peer reviewed and source controlled. Store this either alongside the code, or in an "overarching" architecture repository. Ensure your CI/CD process automatically publishes the latest documentation when it changes. Finally, make sure the documentation you are creating has a purpose and stays up to date, if it's old or no longer serves a purpose - archive it!