Updates to announcing Reolink doorbells on Alexa

A while back, I wrote a post on how I build a Node-Red flow to allow me ask specific echo devices to announce visitors pressing my ReoLink doorbell. This worked great, but there were a couple of shortcomings; 

 

• The list of registered devices would last only as long as Node-Red was active. If I needed to restart the Node-Red docker for any reason, I would also need to re-register announcement devices. 
  
  
• The ONVIF node the flow relies on is a little flaky, and will occasionally just disappear from Node Red. 

I've added functionality to store the list of devices in a text file, so it persists between Node-Red restarts. I also added some notes on how to recover from the self-destructing ONVIF nodes, bring this all up to v1.1.1.

The device list saving is managed automatically as part of the voice registration process, but there's also a manual trigger that can be used in testing. If you use this, you might need to adjust the save and read file locations, particularly if you're not running Node-Red in a docker, like me.

The full flow is pasted below;

VirtualHere on piCorePlayer

This is niche...

I'm in the middle of building a DIY music streamer based around piCorePlayer on a Raspberry Pi with a HiFiBerry Digi+ Pro HAT. This will live in my main AV rack and output to my AV Processor.

I selected PiCorePlayer as I already run a Logitech Media Server (LMS) and have a few squeezebox devices, so that OS is a good fit for me.

I also have an Elgato StreamDeck that I purchased some time ago that I'd like to use in my Home Cinema. This device has a bunch of customisable buttons that I intend to configure with shortcuts for lighting, av system power etc. 

The issue I have is that the StreamDeck needs to be plugged in to a PC via USB for configuration and operation. I don't have a PC nearby, and would ideally like to manage it from a Virtual Windows machine, but that runs on my Unraid server in a different building.

It would be great to be able to use the Pi in my DIY streamer to somehow relay the StreamDeck to my VM.

I came across some VirtualHere software that appeared to do just that. This runs a 'server' on a computer that you can plug USB devices into, and a 'client; on a remote machine. The client can connect to the USB devices as if they were local. 

There is a linux/arm server for PIs available, and a Windows client. The problem is how to install it on PiCorePlayer, which is based on Tiny Core Linux (TCL), and while it supports extensions and the like, is fairly locked down, and ultimately runs in RAM with no persistent storage out of the box.

It took a while, but I finally came up with a somewhat bodged solution.....

Migrating Deprecated Unifi Docker on unRaid

I've been using LinuxServer's Unifi-Controller docker for a long time to manage my unifi devices. However, this has been deprecated and to keep up with new releases, there is a requirement to install a Mongo database docker and a newer Unifi docker. This sounded like a bit of a faff but I thought I'd give it a try before succumbing to a physical controller device. The process did not go well, and I couldn't seem to get the systems to work together.

However, in searching around for solutions, I saw there was a new 'all in one' option, the Unifi-Controller-Reborn docker from PeteAsking. (find it in the Apps tab). I gave it a whirl and was able to get my old setup running in a shiny new Docker. Along the way were a few learnings that I thought I'd log here for posterity.

Backing up unRAID to Stabelebit Drivepool using Kopia

Since I retired Mediaserver 8.1 and  Mediaserver 8.2 in favour of Mediaserver 8.3, I've had a spacious ThermalTake Core X9 case and AMD FX-8320 based motherboard sitting around. I also have a selection of hard drives ranging from 500GB to 4TB that are retired from my main unRAID system. I'd been hankering to turn all of this into a backup server and, in fact, had made a few attempts that never really made it past proof of concept. It was time to attack the project seriously....

Building an UnRaid Home Server for Media Storage and Automation

Are you looking to create a versatile and efficient home server that can handle your media storage needs while seamlessly integrating with your home automation system? If so, building an unRaid home server might be the solution you've been searching for. In this post, we'll explore the benefits of unRaid and provide practical advice on building your own home server for media storage and automation.

I've been using unRaid for home storage and automation for about 10 years now, and have been blogging about it here for almost as long. A lot of my posts are quite specific, so I thought I'd try a high level guide for those starting out. Let me know what you think, or if there are other specific topics you'd like to see covered.

Tidal not playing through Squeezebox with LMS in Docker

Starting to rebuild my whole-house audio playback system based on LMS as Roon, while nice, is not as free as LMS. Plus, I can get my LMS to access and play not only local files and Tidal, but also Spotify, which is missing from Roon.

Anyway, in getting back into this, I came across a frustrating issue - I could not get Tidal to play on any client device.

There's a few posts out there on things to check, like Tidal plugin setup in mysqueezebox.com, use FLAC file format etc., but nothing worked for me.

I had a look in the LMS server log file and came across a bunch of these errors;

fh=Slim::Networking::Async::Socket::HTTPS=GLOB(0x563293263068)
[23-03-07 11:43:22.4710] Slim::Networking::IO::Select::__ANON__ (130) Error: Select task failed calling Slim::Networking::Async::HTTP::_http_read_body: Error in tempfile() using /config/cache/tmp/XXXXXXXXXX: Parent directory (/config/cache/tmp/) is not writable at /lms/Slim/Formats/FLAC.pm line 950.

I run my LMS as a docker in unRaid. When I checked my mapped paths in the docker, I see 'config' maps to my appdata/LogitechMediaServer/. Inside there, I used chmod via SSH to set permissions on the /cache/tmp/ directory and instantly the Tidal stream started right up.

Mapping additional and network drives inside an unRaid Windows VM

On the unRaid forums recently, I was responding to a user who was asking why his VM gcow2 image did not shrink when files are removed (thread here). I advised that auto-shrink is not standard behaviour, and in the discussion, it became clear that the use case was security camera footage. The user figured out that perhaps saving video files to a network share would be better than to the VM boot / system drive, and I mentioned that I do this with my BlueIris setup. I promised a quick guide, so here it is....

User Presence Detection using Node-Red and Unifi

A big challenge in Home Automation is implementing a reliable mechanism for presence detection - figuring out who's home, and tailoring logic accordingly.

Here's how I do it using Node-Red, Unifi and MQTT.

Many approaches to presence detection rely on interacting with the device itself, either by way of software running on the mobile handset, or having the mobile ping or poll an external service. None of these approaches really appealed to me as they are fiddly, rely on handset configuration & maintenance and need to be reset every time someone gets a new phone.

The approach I've taken here is to rely on the fact that all the families' mobile phones are configured to connect to home WiFi (to save on data charges!). This method interrogates the home network to see what phones are present - a good indication that the person is home.

In short, I have a Node-Red flow that polls the network on a schedule, looks for named devices, and updates presence status in MQTT based on that.

I could have used a database, or flat file storage for storing state, but as I have an MQTT broker running anyway, it's a convenient and flexible approach for me.

Problems passing through GPU to an unRaid VM?

One of the most common problems I see people seeking help for on unRaid forums, reddit and discord is the thorny issue of GPU pass-through. People with the same problems tend to ask the same questions over and over, and, somewhat tired of providing the same troubleshooting steps over and over, I thought I'd write up the main pitfalls, and possible solutions.

Here are some steps to check before reaching out for help.

1. Have you done your research?

As noted, there's a ton of threads on unRaid forums covering this, many successfully resolved. Do conduct a search there. For example, searching 'problem with gpu passthrough' today produces 40 pages of results. Chances are, your issue has been raised and resolved several times before.

Another oft-posted remark is something along the lines of 'I followed spaceinvaderone's video, and it still doesn't work'.

In most cases where I've seen this, the user has incorrectly followed the video - missing a key step or subtle point. This is NOT a straightforward process. There are many variables involved and SpaceInvaderOne's videos are packed with information. He does an amazing job of explaining the complex process step by step, so make sure to rewatch these videos several times to ensure you haven't missed anything.

If you haven't been following one of his videos for GPU pas-through, why not? Get very familiar with the process and follow along before posting queries;

Here are some links to his channels;

https://forums.unraid.net/profile/67288-spaceinvaderone/

https://www.youtube.com/channel/UCZDfnUn74N0WeAPvMqTOrtA

https://www.reddit.com/user/spaceinvaderone/

Research done? Let's look at some of the common missteps....

Solving the -43 issue for GPU passthrough

I've usually built my VMs with AMD GPUs when passthrough was needed. I happened to have an old GeForce GTX770 lying around, but found it frustratingly difficult to get it working in a Windows 10 VM. nVidia drivers apparently really, really don't like running in a VM and cause the display device to show a -43 error.

For me, after a significant amount of tweaking and trying, I finally found a combination that worked;

Machine Type: Q35 4.2

Bios: OVMF

Graphic ROM Bios (techpowerup download, hexedited to remove header)

And the following edits to the </features> section of the XML. ˜The highlighted lines are added in addition to the standard unRAID config;

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

      <vendor_id state='on' value='1234567890ab'/>

    </hyperv>

    <kvm>

      <hidden state='on'/>

    </kvm>

    <ioapic driver='kvm'/>

  </features>

MediaServer 8.3 - Bifurcation Edition

Some hardware updates, some minor mods and lots of cabling later, I've managed to get my unRAID server hosting eleven (13?) usable PCIe/PCI expansion cards which I'm using to manage storage devices and populate VMs with passthrough GPU, USB and Utility adapters.

All with 2x slots to spare on my 5x slot mainboard.

Here's how...

unRAID discrete USB Passthrough to multiple VMs

For a while, I've been working towards consolidation of machines into VMs on my unRAID system.  unRAID is great for this as it allows for hardware passthrough of devices such as GPUs, USB adapters and other items. Passthrough of discrete USB controllers is especially important. unRAID does allow direct device passthrough, but I've found this unsatisfactory for a number of reasons. Mainly, it's not a great solution for high-bandwidth devices such as audio cards. Also, it's not really plug'n'play, with hot swapping being somewhat patchy.

Passthrough of the full controller solves these problems, One of the challenges of this approach is however, physical capacity for discrete devices. With each VM requiring a PCIe GPU and a USB adapter, it's possible to run out of slots fairly quickly. Yes, it's possible to use on-board controllers, but that's not always a full solution and PCIe cards are often required in addition.

Another approach is to use multi-controller PCIe USB adapters. These are cards the occupy a single slot, provide multiple ports and. crucially, have multiple on-board USB controllers that would, in theory, permit each controller to be passed through to a separate VM.

One such card is the Sonnet Allegro USB3-PRO-4PM-E, sadly now discontinued. There are alternatives, but as users in this thread are finding out, broad compatibility is patchy.

As an alternative, I've been using a PCIe expansion kit to pass through discrete controllers to VMs. These devices were all the rage during the mining boom as they allowed users attach multiple GPUs to a single PCIe slot in order to leverage compute power. They work by essentially switching multiple PCIe ports through a single 1x motherboard port. Although the device sports four 16x mechanical slots, each is in itself only a 1x slot - but that's all that's needed for a USB controller.

Splashtop Latency Demo

A user over at unRaid Forums asked me about latency when using SpalshTop Personal to access an OSX VM running on unRaid. I promised a quick video to show it in action. Sorry about the delay!

Enabling HEVC on Rx570 in MacOS VM

In following the MacOS Bare Metal thread on unRaid forums, it was bugging me that I wasn't getting full performance from my VM GPU. I have a Sapphire RX570 4GB passed through to a Mojave VM, and while it was showing Metal and H.264 encoding support, HVEC remained resolutely elusive.

It should be simple, enable Lilu and WhateverGreen kexts and go. It turned out to be anything but.

Video Surveillance on unRaid

Inspired by everyones favourite unRaid you tuber SpaceInvaderOne's then partially completed series on setting up CCTV in unRaid, I figured it was time to get started on a little bit of home security.

I'd previously worked on building an interface to my legacy Comfort alarm / home control system, and that's fine for basic alarm, but video was always the next step.

SpeaceInvaderOne conducted a basic analysis of cameras and software available, before publishing a walkthrough of how to set up his chosen Shinobi docker. This, along with some other research, started me on my journey. I went ahead and purchased two of the SV3C 1080p PoE cameras he mentioned. These would connected to the spare PoE ports on my Unifi Switch 8 (see my Unify blog post for more details).

I tried installing Shinobi, (before the walkthrough dropped), and failed miserably. In any case, I'd previously researched the area and at the time decided that Blue Iris looked like the best option for scalability and integration.

VM Audio Solution

When setting up VM access over ethernet using the AV Access device, I noted that one issue was the failure of my Harman Kardon Soundsticks to work with the system. Plugging them in to the receiver USB port would result in intermittent but regular screen blanking.

A few readers suggested additional power and I purchased a USB power/data cable, but that didn't work.

Next, I tried a cheap and cheerful external USB soundcard. I plugged this into the receiver and attached some powered speakers. This worked, in that my OSX VM recognised the device immediately and could play audio, but it was annoyingly distorted - just enough crackle & pop to distract.

Next, I tried attaching the USB soundcard directly to the (passed through) USB adapter on the unRaid box. I then passed the audio out to the AV Access transmitter audio in and attached the speakers to the receiver audio out.

Hey Presto, perfect audio*. No crackles, pops, noise or distortion. The lesson here is that with this method of video/IO/audio distribution, the soundcard needs to stay upstream. My SoundSticks are now moved to another workstation where they can be attached to that VMs USB adapter directly.

My final working audio chain is;

PCIe USB Adapter -> USB Audio Card -> AV Access TX Audio In -> AV Access RX Audio Out -> Speakers

*When I say 'perfect', I mean adequate for needs in that system sounds and youtube audio plays just fine. I wouldn't be using this set up for anything close to critical listening.

MediaServer 8.2

It’s been over 5 years since I built MediaServer8, an unRaid based system that looks after not only storage duties, but also hosts my main workstation, manages whole-house audio systems, whole house LiveTV and PVR, home automation and a bunch of other stuff through dockers, VMs and plugins.

The AMD FX8320 based system has been rock solid in all of that time. Indeed, the only problem I’ve ever had occurred in the past week when my dual 1TB spinners assigned to a cache pool started showing millions of unrecoverable BTRFS errors. Fortunately, I rescued my appdata and domains folders in time. My docker image was not so lucky :-(

A few months ago, the itch emerged and, try as I might, I found it difficult to resist the urge to scratch. There was nothing wrong with the system per se, but with ‘only’ 8 cores and 32GB RAM, it was a bit restrictive in the number of VMs I could run simultaneously, and no real scope for growth. I had a particular hankering to virtualise the dual boot Hackintosh the family use daily, as well as a few other utility systems. Threadripper was calling. Here’s how I answered....

Routing VMs anywhere

I'll start my saying that these things are freaking amazing!

I've been running a few VMs on my unRaid box for a while now. Specifically my main workstation (Win10 /OSX switchable), as well as some utility boxes.

One of the challenges has been physical location and signal transfer - my unRaid server resides in my attic and any VMs that required an interface (display & USB), would need to be located within the reach of a HDMI run.

This ruled out virtualising some systems, including a Win 10 / OSX dual boot hackintosh that's the main system the family uses. It's miles away from the server with no hope of running cables.

I'd toyed with HDMI over ethernet before, with little success. Recently, however, I came across the Av Access range of products...

OpenVPN via tethered connection - no root, no PC software installs

A while back, I'd installed an OpenVPN docker on my unRaid server to allow me access home systems remotely. It worked for a while, but recently I'd been unable to use it as a lot of computers I use away from home are locked down by corporate IT.

I'd installed OpenVPN client on my Android phone, thinking I could tether from a PC and access home systems that way. No dice. It turns out Android specifically restricts tethered connections accessing VPN connections due to security concerns. Workarounds are either root the android device, or install some kind of proprietary tethering. The first I didn't want to do, the second I couldn't as the PC does not permit software installs.

I was about to give up when I came across Every Proxy, a very simple android app that installs a https or SOCKS proxy server. Now, when I tether my android to PC (WiFi or USB), I can enter the proxy details in the browser config. and, hey presto, the PC can access the OpenVPN tunnel to my home server and I can access unRaid or any of my VMs (via vnc).


Yay!

UnRaid Not Accessible from Windows 10 Explorer

I recently updated a Windows 10 PC to the latest and greatest through auto updates. (currently Windows 10 Pro Build 1803). After doing so, I found I could not access UnRaid either by name or IP address. (Windows explorer was showing an unspecified errorcode Ox000 )

After a bit of GoogleFu and some trial and error, the following steps worked for me;

1. Run services.msc and set Function Discovery Resource Publication Service to automatic(and start it).
2. Enabled SMB 1.0/CIFS File Sharing Support under Control Panel ->Programes & Features -> Turn Windows features on and off.
3.  Run gpedit.msc and set Enable insecure guest logons to enabled (under Computer Configuration -> Administrative Templates -> Network -> Lanman Workstation.

I'm unsure if all of the above are necessary, but 3 was required for me after 1 & 2 solved the basic error. There may be other ways to solve this, the above worked for me.