Installing Home Assistant, Mosquitto & AppDaemon on Debian

As part of some hardware upgrades, I’m moving Home Assistant and it’s associated programs off it’s shared box onto a dedicated VM. This means a fresh install of everything. Whilst docker would usually be an ideal candidate for this, I decided against it because I use a few custom components and troubleshooting these is a lot easier without Docker.

Preparing the VM

Nothing too special here, I used the latest network install media for Debian Buster (10). The netinst media is particularly useful if you have a fast internet connection, as the latest version of each package is downloaded before installing). Unfortunately this does mean slowing internet connections or offline installs are out of the question. If you’re doing a large number of installs, a local debian repository should work with the netinst media, but I have yet to test this.

Once this was done there’s the usual routine of adding SSH keys (yay for Ansible), setting up backups (Veeam works nicely), and installing Monitoring (Zabbix is awesome). I didn’t pay much attention to the system Python version, as Debian still ships with 3.5 which Home Assistant no longer supports.

Installing Dependencies

There are a number of global dependencies we can install through Apt to save time in the future. Here’s the command I eventually came up with to get everything up and running

Installing Home Assistant

By this point, I hope I don’t have to explain what Home Assistant is. In short, it’s some of the best home automation and BMS software out there. Unfortunately the latest versions have some dependency issues with AppDaemon (despite being designed to work together) and has now dropped support for Python 3.5. Because of this, we’ll install a separate version of Python for each of them.

Create a system account

First, we’ll create a new user to run the service. Useradd can do most of the work for us, using these parameters:

  • -r Create a system account, disables password ageing and uses a lower ID
  • -m Create the user’s home directory
  • -d Specify the location of the home directory
  • -s The login shell (or lack thereof) for the user

Overall, that gives us this:

From here on, everything will be run from this user account, because nothing else needs to use it’s version of Python or the libraries. To make this easier, we can use sudo to run bash as Home Assistant’s account:

Install Python

It’s always handy when people scripts exist to do the work for you, and installing Python is no different. To handle the installation, we turn to PyEnv. Using these commands we can install PyEnv, and then use it to install the latest version of Python.

Install Home Assistant

Now we have a working install of Python, Home Assistant can be installed using it’s version of pip

Starting automatically with systemd

There are lots of arguments in favour of and against systemd, though from my point of view the benefits outweigh the disadvantages. For example, here we’re going to tell systemd to start home assistant when the system boots, but after the network and Z-Wave adapter are ready this was causing a few issues for me, so I’ve removed it for now.

Installing Mosquitto

Out of the three Mosquitto is by far the easiest to install, and that’s saying something

And that’s it. Told you it was simple. As with all of these, the configuration will need adjusting to suit your needs, but that’s well covered by the relevent documentation.

Installing AppDaemon

AppDaemon is a companion for home assistant, and is capable of doing much more powerful automations using python modules & scripts.

The first part of this is very similar to the Home Assistant installation. I’m doing them using separate Python versions and environments because there are some dependency issues between the two. Keeping the environments completely separate will make updating in the future much easier.

Create a system account

Same as for the Home Assistant user, Debian will do most of the work for us with this command:

Most of the following commands needs to be run as the appdaemon user. We can switch to it using sudo to explicitly run Bash

Install Python

Again, we’ll use pyenv to install the latest version of Python for the service account

Install AppDaemon

Again this is fairly simple. AppDaemon is available through PyPi, so it can be installed using the Pip command:

Starting automatically with systemd

Finally, we can use systemd to automatically start appdaemon with the server. The below unit file can be placed in /etc/systemd/system/appdaemon.service

There’s a couple of non-obvious options in here, so let’s go through them all:

  • After… – This makes sure appdaemon doesn’t start until Home Assistant is up and running
  • ExecStartPre… – Like many users I store my Home Assistant configuration in Git. AppDaemon requires permissions on it’s configuration file, and modifying them through git as another user may accidentally remove the permissions. This command explicitly grants those permissions each time AppDaemon starts.

Installing/configuring HASS Apps, or any other AppDaemon app

One of my main uses of AppDaemon is Schedy, part of HASS Apps. Though this is just an example, and should work for any AppDaemon-powered app. First we install HASS Apps:

Now we can configure AppDaemon, using the config file /var/lib/appdaemon/config/appdaemon.yaml and it’s matching secrets file.

Then we need to load the HASS Apps module into AppDaemon. This is done using a shim in /var/lib/appdaemon/config/apps/hass_apps_loader.py

Now the important bit, we tell AppDaemon to use Schedy and what schedules we want. This goes in /var/lib/appdaemon/config/apps/schedy.yaml

Turning Everything On

Now all of our software is in place, and has a basic starting configuration, let’s tell systemd to turn everything on!

Home assistant will take a while to start for the first time, so now is ideal to go and make a cup of tea. In my case, this took over 90 minutes due to the complexity of my particular setup.

Accessing Multiple Services over HTTPS via SNI Proxy

Like most people (OK, maybe not most…), I have a range of locally hosted services that I like to access remotely. Because of the joy that is IPv4 depletion (long live IPv6) I’m forced to use different ports for internet access, which is hard to remember and annoying to set up. And because the internet is a bad place, I prefer to use encryption where possible to keep everything safe on it’s journey across the internet. Sure, something like SSH tunnels would work, but that would still require remembering the different port numbers, and isn’t compatible with all devices (such as mobile clients).

Enter SNI Proxy

After some research, I decided to put an end to both these problems, with an install of SNI Proxy. SNI Proxy is a generic HTTP and TLS proxy that identifies the internal host from TLS’ server name indication (SNI). This allows it to seamlessly proxy multiple services on the same port, and heavily increase the WAF for services like https://coffeemachine.jacobmansfield.co.uk and https://photos.jacobmansfield.co.uk . It also makes a nice place to centrally manage SSL certificates and Let’s Encrypt renewals I was under the impression that SNI Proxy could also terminate an HTTPS/TLS connection, though this now appears to be incorrect. (Though I do this on each host, as SNI Proxy doesn’t handle my IPv6 traffic). So, let’s get installing.

Continue reading Accessing Multiple Services over HTTPS via SNI Proxy

Adding Android Widgets Without User Interaction

As part of an ongoing project, I’m building a simplified replacement for Trebuchet (Android’s default launcher). One aspect that I need to simplify is showing Widgets on the launcher without requiring the user to add them by hand. Normally, Android restricts this by requiring apps to use the  AppWidgetManager.EXTRA_APPWIDGET_ID  intent to launch a system picker. The only way around this is for system apps to request android.permission.BIND_APPWIDGET in their manifest, so this method is only available on rooted devices where the app is installed at the system level (in  /system/app  or /system/priv-app). Continue reading Adding Android Widgets Without User Interaction

Fancy Android Version Numbers from Git

I recently started working on several Android projects, including a custom automatic updater (the target devices won’t have internet access, let alone the Play Store). While the system is working wonderfully, it relies on detecting changes in the app’s version number. Which I keep forgetting to change. As with most of my projects I’m using Git for version control. So let’s do something fancy, and generate the android version number from git’s commit and tag information. Continue reading Fancy Android Version Numbers from Git

YRS Festival of Code

Young Rewired State Festival Of Code 2015 LogoThe pride and joy of Ruth Nicholls, the YRS Festival Of Code, starts up again next week. With coding opportunities for under 18s at 66 different locations across the country and beyond, there’s no reason anyone technically inclined can’t take part. Personally, I’m mentoring in a centre at Redgate Software, in Cambridge, helping 20 young coders, designers & developers find their calling, and home their skills.

The centres include an online ‘virtual’ centre, and ones in Times Square, New York, Bern, Switzerland, and Prishtina, Kosovo (that last one is in the south of Serbia, I had to look it up). There’s also loads of other centres across the country, so there’s always that’s convenient to attend.

The week is spent building apps, websites, games, and even hardware-based hacks, inventions, and other crazy contraptions, all involving open data. At the end of the week, all 1,200 contestants meet up in Birmingham, where they present their projects to expert judges, mentors, the press, and other participants. The best entries from each group will go through to the finals, where prizes are awarded for the a range of different categories. This year’s prizes haven’t been announced yet, but rumours say they include Amazon and Pimoroni gift vouchers, an awesome-looking 3D printer, a quadcopter drone, and even a McLaren F1 experience! This year’s categories include:

  • Best Example of Code
  • Best Example of Design
  • The ‘Should Exist’ Award
  • Code a Better Country
  • Best in Show
  • The People’s Choice

Unfortunately sign-ups for centres closed back in July, but if you’re a participant, a mentor, or a volunteer, you’ve still got time!

Installing Skype on Debian Jessie

Skype LogoGoing to keep this one quick, here’s a quick guide for installing Skype on Debian Jessie. This is based on version 4.3.0.37, but should just need a URL switch for newer versions.

From there, you should be able to log in and chat away. Details on autostart (on login) to follow.

Shrinking Raw Disk Images

I recently had a Raspberry Pi image that I wanted to deploy to several Pis (is that the right plural?) for production use. During development of this, I worked on a 32GB SD card for some breathing room, but to save costs, we decided to deploy onto 4GB cards.

This left me with a small issue. Although the file system on the SD card was only using 2.5GB of space, any image files from it were still 32GB, far too large to go onto the SD cards. I need to remove the white space from the end of the image to make it fit. So, here’s a quick tutorial on shrinking raw disk images.

Continue reading Shrinking Raw Disk Images

Testing for SSL Vulnerabilities

2015-04-28 16_15_44-SSL Server Test_ jacobmansfield.co.uk (Powered by Qualys SSL Labs)
Qualys Labs Server SSL Test for jacobmansfield.co.uk

So around now is the annual SSL certificate renewal for most of our internal servers, and I thought it would be a good idea to check them all for SSL/TLS vulnerabilities. A quick Google later, and I’m looking at Qualys Labs SSL Server Test. The scan is nice and shiny, and give an instant (2-3 minute) overview of a server’s security. It also takes care of DNS round-robin to make sure all your servers are handled (though sites with HLBs or similar may need more attention).

Continue reading Testing for SSL Vulnerabilities

Non-matching records in T-SQL One-to-Many Relations

I was recently tasked with finding all the people in our CRM software that lack email addresses, and with around 3000 contacts, there was no way I was going to do it by hand. SQL to the rescue!
There was just one issue, email addresses are stored in a different table to allow each person to have multiple addresses on file. After a bit of digging around, I came up with this:

Let’s go through this line-by-line so we can see exactly what it does.

Continue reading Non-matching records in T-SQL One-to-Many Relations

SCCM & SQL

Today I decided to install SCCM to see if it would help with the WDS work I have been doing.
After a quick download from the Microsoft website, I kicked off the installer, only to find a nice helpful error message:

Continue reading SCCM & SQL