When running this smoke test, it will fail intermittently with the following screen after making setting changes:

Error:

I suspect that this is due to the fact what we are writing to the .cfg files, which reloads the bundle, which makes the service temporary unavailable.