Dave Dupplaw Full Stack Developer http://david.dupplaw.uk/ Making stuff work... most of the time.

Linux Mint 20 Installation Recovery

I decided to update my old Linux Mint 18 installation to the latest and greatest. The docs seemed pretty straightforward, so off I went.

I first had to get to 18.3, then to 19, then to 19.3 then 20. The process for each was pretty similar. Sadly, it didn’t all go to plan.

Read more...

Seek to Last-But-One Message in Spring Kafka

One of the use-cases for Apache Kafka is to use it as a storage layer for an application. You are able to rewind and fast-forward through messages which makes it behave like data cache. If the expiry for a given topic is set to forever then it behave like a persistent data store.

However, when you’re using Spring Kafka and much of the internals of the Kafka system is hidden from you, then some of the more complex situations can be harder to achieve.

Read more...

RAID Lost! Recovery on Thecus NAS

In these digital days, a family ends up with so much digital stuff that you need the equivalent of a junk room to put it all in. That’s what a NAS is for us. Unfortunately, half of that junk was unceremoniously wiped out the other day and I wanted to get it back.

This is what I found:

Read more...

GitLab CI Docker Compose Integration Test Artifacts

If you followed my previous post on Integration Testing With Docker-Compose you may have gone the next step and tried to implement this within your CI pipeline.

I use GitLab and usually set up my system the same as the gitlab.com system, where jobs are run in Docker containers. When GitLab is running jobs in Docker containers and those jobs run docker-compose to instantiate Docker containers: it all gets “a bit Inception”.

So let’s look at how it all works.

Read more...

Cucumber List of Strings Parameter Type in Java

I’ve recently been adding some integration tests into a project I’m doing at work. We use Cucumber to write our high-level tests and run the tests against a Docker Compose network spun up during the GitLab build pipeline (at the moment, while they don’t take too long to run).

I wanted to write a test something like this:

Scenario My Awesome Service gets some data on startup
  Given My Awesome Service is running
  And   It's set up with the test settings
  Then  The data it holds is not empty
  And   The data has 3 parts containing 1, 2, 3
  And   The data for part 1 contains "Kirk", "Bones" and "Spock"

(Obviously not the real test!)

But the last two steps are not possible to generate in a general way, without using custom types.

Read more...

Integration Test with Docker

So, you are all up with unit testing and your code has 100% coverage and 100% mutation test coverage, and you are all great.

However, when you’re working with microservices, you can easily have all your services appearing to work, but once you deploy them all to your staging environment and click a few buttons, you realise these services do not work together.

That’s where integration testing comes in.

Read more...

301 redirects

A 301 is the HTTP response code that tells your browser that the site it visited has been permanently moved to somewhere else.

When moving a site, you can use 301s to automatically forward vistors from your old URLs to your new ones.

The simplest form, is using Apache’s .htaccess files which allow you to control the web server’s behaviour.

Read more...

Flexible CSS with FlexBox

Flex box is the new kid on the block in layout tools for CSS. It’s come from a combination of all the various existing grid technologies that were necessary due to the lack of anything like Flex box. The good news is that it’s now widely supported enough to be used on pretty much any website (unless you intend to deploy to Opera mini users). Another advantage is that, because it’s implemented at a browser level, it’s noticeably quicker than CSS ‘hacks’.

So, I’ll show you here a quick overview of flexbox and how to make a simple site skeleton with fixed header and footer and proportionally stretching panels inside.

Read more...

Christmas Handbell App for Groups

Quite a few years ago now, I purchased a pack of Christmas crackers that contained small handbells, as well as a baton and some music. A bit like these. It was great fun at Christmas dinner trying to recreate some Christmas carols.

Later on, I purchased some bigger kids’ bells that were a single octave chromatic scale (like these). I also purchased an extension to extend the range to an octave and a half - and they took nearly a year to arrive!

My Set of Bells

I teach a music workshop once a month and when it comes to Christmas we usually play some carols and recap the things we learned during the year. One year I took the bells and we had a lot of fun trying to make music with them. With so many bells, it’s a bit easier when there are 12 people to have 1 bell each!

Read more...

Webpack and Typescript for Google Cloud Functions

I’ve been doing some work recently with api.ai and looking at ways of making more interesting chat bots using their fulfilment functionality. This functionality allows you to call out to services to provide the processing for a user’s specific intent.

One way to deploy such a service would be on Google Cloud using their Cloud Functions. These are tiny pieces of Javascript (node) code which performs a function. They are true microservices and Google provide the entire platform (Platform as a Service [PaaS]).

Google’s online tools do allow you to code directly on the Cloud Console, but it’s pretty awkward and the code has to be vanilla JS. So I wanted to find out how easy it would be to make a more maintainable cloud-function codebase using Typescript.

Read more...

What3Words and Which3Words

Addresses are long and tiresome to remember. And many things don’t have addresses.

“Meet me by the lake. Near the bench on the side with the bridge. The longer bridge - the one with the knobs on.”

“Which lake?”

What3Words is an attempt to overcome this failing of our addressing system.

Read more...

GitLab CI with Docker

One of the reasons I moved to GitLab is that it provides free private repos as well as free CI. It’s CI is based around Docker images which makes it dead easy to build any sort of project from GitLab.

I thought I’d share with you here the simple configuration I use to build things (such as this blog) and put it on to my web-hosting.

Read more...

Clarity Icons

We all love icons. Icon shopping has a similar effect on me than walking around Staples does. There’s a stationery buzz.

Font Awesome is the ubiquitous, go-to set for web-icons these days, but it’s still a bit fiddly to get it into a Typescript/Webpack project needing sass imports, web-pack loaders.

So, there’s always a bit of excitement raised in me when I find a new source for fonts or icons. And today I found Project Clarity.

Read more...

Typescript Maps

Unlike ES6, Typescript doesn’t have its own implementation of maps.

In ES6, you can instantiate maps pretty much the same way as in Java, except you can then use set and get methods to, er, set and get values.

let map = new Map<string,string>();
map.set("key","value");

Read more...

Running stand-alone Ruby script in a Rails Environment

I found out today how to have small stand-alone ruby scripts that run in a specific Rails environment.

Read more...

Trello Card Label Extension

The Card Color Titles for Trello is a must-have Chrome extension if you use Trello at all.

The labels that you add to cards show up as little colour bars by default and I’ll be damned if I can remember what all the colours mean.

Default Trello Labels

Read more...

SVG Icon Resource

It can always be a bit of a faff to find decent icons when you’re building something. Font Awesome is an obvious go-to resource for some icons, but sometimes they’re just not enough or they don’t have the right ones.

Read more...

My First SmartThings App

The automated home has not yet reached ubiquity, but systems like SmartThings are certainly making it easier to jump into the fun of controlling the phsyical world; even more so when Google Home and Alexa all come with SmartThings integrations.

As a dad to 5 and 7 year old boys I often find lights left on around the house and when those lights are upstairs there was a time I’d have mumbled a few choice words and thudded my way up the stairs to turn it off (surely children should have evolved to understand how high energy prices are now?). But no longer!

Read more...

Redis + Search = RediSearch

Just came across this nice looking project: RediSearch. It adds on top of Redis a full-text search engine capable of incremental indexing, fuzzy searching (for autocomplete) as well as numeric range and geo searching. It has a bunch of libraries ready to go in various languages too. Looks to be an interesting alternative to heavier weight frameworks like ElasticSearch or Solr.

Read more...