Mocking RESP API in 20 minutes via Yakbak

Imagine this: you are an ordinary frontend developer. When you open your mailbox you found a message — tomorrow DevOps team will make an optimization with Kubernetes. You are experienced developer and you know that environment operation test might go sideways. Test environment is crucial for your job as frontend developer and you don’t want to miss a whole day on a job, so there are two possible solutions present:

1. Setup all microservices on your laptop
2. Prepare mocks for API

I will describe how to mock REST API request via Yakbak.

Demo Application

In order to demonstrate the mocking process for application I created the application that take a name of Lord of the Ring character and check Github’s user with this login and Github’s commits, that have this name.

• I generated Angular application via Anglular-cli;
• The list of characters I found on the the-one-api.dev;
• I will launch user search and commits following instructions from Github docs.

The application's logic is very simple. User select a character and application call GitHub REST API. We will display response via beautiful RPG UI.

If you want to see application itself, please click here.

Mocking API requests

Developer runs yarn start and application starts at localhost:4200. Browser sends API requests to the local proxy that starts on the same address. Obviously I can send requests directly to Github and the proxy needed only for demonstration.

Proxy config:

const PROXY_CONFIG = {
    context: [
      "/users",
      "/search"
    ],
    "target": "https://api.github.com/",  
    // "target": "http://localhost:3111",
    "changeOrigin": true,
    "secure": true
}

module.exports = PROXY_CONFIG;

This picture describes how application works without Yakbak cache:

And this is how application will work with Yakbak cache

I used Yakbak library for the preparing the cache layer. Here is how this library work:

1. Library get a request and calculates a hash of them;
2. Library checks if a file with that hash is in a specific directory;
3. If file does not exist Yakbak sends request to target server and received response writen to a file. Afterwards it responds to original request;
4. If the file exists then Yakbak reads the file and responds to the original request.

For use this solution you need to switch target in the proxy config from https://api.github.com/ to http://localhost:3111 and run node yakbak-conf.js record. After that all new responses to the Github will be recorded to the tapes/ folder. But if you will run command node yakbak-conf.js (without the record word) Yakbak will only respond with files already written and will not send new requests to Github.

If you will use this solution for a while you will realize that it's better to use readable file names and not necessary put all query parameters to hash. That is a reason I selected the following hash format: ${req.url}__${req.method}__${queryHash}. In addition, it is always convenient to open recorded file and correct something there as you see fit.

The list of cached files:

...
__search__commits__GET__eyJxIjoic2F1cm9uIn0=.js
__search__commits__GET__eyJxIjoidGFudGEifQ==.js
__search__commits__GET__eyJxIjoiZnJvciJ9.js
__users__adaldrida__GET__e30=.js
__users__adalgar__GET__e30=.js
__users__adalgrim__GET__e30=.js
__users__adamanta__GET__e30=.js
...

var path = require("path");

/**
 * GET /users/adalgar
 *
 * cookie: Webstorm-7f35e0f0=5cbe6ea7-fbdc-4756-42-b67baf0b74c4; AIOHTTP_SESSION="gAAAAABh4C__bz3aTNe_Podsmatrivay_kq3iCaA=="
 * accept-language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7
 * accept-encoding: gzip, deflate, br
 * referer: http://localhost:4200/
 * sec-fetch-dest: empty
 * sec-fetch-mode: cors
 * sec-fetch-site: same-origin
 * sec-ch-ua-platform: "Linux"
 * user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36
 * sec-ch-ua-mobile: ?0
 * accept: application/vnd.github+json
 * sec-ch-ua: "Chromium";v="106", "Google Chrome";v="106", "Not;A=Brand";v="99"
 * cache-control: no-cache
 * pragma: no-cache
 * connection: close
 * host: api.github.com
 */

module.exports = function (req, res) {
  res.statusCode = 200;

  res.setHeader("server", "GitHub.com");
  res.setHeader("date", "Wed, 16 Nov 2022 21:26:23 GMT");
  res.setHeader("content-type", "application/json; charset=utf-8");
  res.setHeader("cache-control", "public, max-age=60, s-maxage=60");
  res.setHeader("vary", "Accept, Accept-Encoding, Accept, X-Requested-With");
  res.setHeader("etag", "W/\"07c85ce555cf40c22d222e0bfaccc7cccf5528d40312724324341b02e19ba0\"");
  res.setHeader("last-modified", "Sat, 06 Apr 2019 16:00:30 GMT");
  res.setHeader("x-github-media-type", "github.v3; format=json");
  res.setHeader("access-control-expose-headers", "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset");
  res.setHeader("access-control-allow-origin", "*");
  res.setHeader("strict-transport-security", "max-age=31536000; includeSubdomains; preload");
  res.setHeader("x-frame-options", "deny");
  res.setHeader("x-content-type-options", "nosniff");
  res.setHeader("x-xss-protection", "0");
  res.setHeader("referrer-policy", "origin-when-cross-origin, strict-origin-when-cross-origin");
  res.setHeader("content-security-policy", "default-src 'none'");
  res.setHeader("content-encoding", "gzip");
  res.setHeader("x-ratelimit-limit", "60");
  res.setHeader("x-ratelimit-remaining", "44");
  res.setHeader("x-ratelimit-reset", "1668635372");
  res.setHeader("x-ratelimit-resource", "core");
  res.setHeader("x-ratelimit-used", "16");
  res.setHeader("accept-ranges", "bytes");
  res.setHeader("content-length", "452");
  res.setHeader("x-github-request-id", "7200:EE5F:2342344:207742:111157F");
  res.setHeader("connection", "close");

  res.setHeader("x-yakbak-tape", path.basename(__filename, ".js"));

  res.write(new Buffer("H4sIAAAAAAAAA52TQYvbMBCF7/kVwefuyukmWWIopdDrLgTaUnoJsqy1VWRJSGOHxOx/70jjTRMfWrwn48e8T29GmmGxXGba1spkxTL7UnFdc599", "base64"));
  res.write(new Buffer("iKqqUFrvHtabx8ddUoyt5CHJ2dPX/fbHz2ctfu83T+f96vksNmTjPQfuD53Xsa4BcKFgjNRwXytourIL0gtrQBq4F7ZlHXs753P/aU2c2o8kOpDEKdWpkUgYxAZ200QDrZ5moQzJcVP7YrW2R0RMDfwfx7CLixLSrzL1OyDoGpiFRuL4sJXXcRAqwLxIdXQMLH7wukZMwFvxspoVa/RgqKPBPAPz0tk3XlcG4ZUDZc28eOHaST1aX3Ojznw+DJ2BGDHbvCDJQWbZ41uc5ybLwJxXPRencSxeCql6nPM7iBMvJYOTk3GTvuOLICUokAdetWlnX7gOkraTt7HQdFqnf3zfjpvTtVTipkcWcbQVaeDXFbLlKi7uhdIoL3mpb8ilstclcFQA45s1kxCuK7USB5p0scxTslFMzxRBJP7do1sFl+JSIzAL4Gw5xC4+5qvdXb6+y7ffVtsiz4uH/Bd11rnqf3WL18UfF7Fg8fwEAAA=", "base64"));
  res.end();

  return __filename;
};

Below is the full Yakbak config, it is quite short and transparent.

const express = require('express');
const yakbak = require('yakbak');
const {pick, omit} = require('ramda');

const record = process.argv.includes('record') || false;
const base64 = (s) => Buffer.from(s).toString('base64');
const getUrlName = (req) => req.url.replace(/\//gi, '__').split('?')[0];
const getQueryHash = (req, queryParams) => base64(JSON.stringify(omit(queryParams, req.query)));

const omittedParameters = [
    'start',
    'end'
];

const yak = yakbak('https://github/', {
    dirname: __dirname + '/tapes',
    noRecord: !record,
    hash: (req, body) => {
        const name = getUrlName(req);
        const queryHash = getQueryHash(req, omittedParameters);

        return `${name}__${req.method}__${queryHash}`;
    }
});

express().use(function (req, res, next) {
    yak(req, res);
}).listen(3111);

That's actually all. All project sources can be found on my Github here.

P.S. I can say that once I used Yakbak to speed up E2E tests for the frontend part — obviously — those tests were not quite E2E afterwards :)