Having spent time recently looking at integrating sprites and volumetric room heatmaps into Dasher, today we move on to a capability that we didn’t have previously but has been enabled by Project Hyperion (i.e. the new Data Visualization Extension for the Forge viewer): planar heatmaps.
Planar heatmaps do just what you’d expect: they display a heatmap for 3D geometry in a plane. This means you can display a heatmap on the floor or ceiling of rooms, or place them on tables or work surfaces. They might – for instance – be used to indicate usage of an area, such as where people have touched a table in a workspace or spent time looking at a display in a retail store, assuming you have computer vision capabilities that can determine such things (an active area of research in our team, by the way). Overall a very helpful mechanism.
Hyperion takes care of a lot of the heavy work of displaying planar heatmaps: it slices through geometry to determine the plane used to display the heatmap and calculates how the 3D sensor points should contribute to the display. It also allows you to display the sensor on the appropriate level inside the 3D space (today this mean’s the minimum or the maximum of the volume, but we’ll talk more about this a bit later).
For the purposes of today’s post, we’re going to look at how planar heatmaps can be used inside Dasher to display sensor data that we typically show volumetrically (we’ll also look at why this isn’t always ideal towards the end, but the main point of this post is to show that the mechanism works and give some idea of how it can be used).
In general, the process needed to create a planar heatmap is the same as for a volumetric one, the main difference being the options you pass into setupSurfaceShading(). Here’s a snippet of the code we use to define our planar options if the displayPlanarHeatmaps flag is set:
Now for a quick word on version support: for Dasher we can only use v7.44 or higher of the Forge viewer to display planar heatmaps, as a number of changes have been integrated by the Hyperion team to support us. The first – and arguably most important one, for now – is the ability to load NWC (Navisworks) models into Hyperion. (This was a simple fix we talked about recently, and it just involved passing an additional “recurse” boolean flag to enumNodeFragments() when analysing the model’s geometry: in this instance when slicing it to determine the boundary for the heatmap. A simple fix, but a game-changer for Dasher!)
This version of the Forge viewer is currently in Staging, but should be released in the coming days. While v7.44 is critical for Dasher, there are other issues that were fixed by the team in this release that would also make using this version important for developers working with a Revit-based workflow. I’ll talk about them in detail below.
In prior versions the sensors in a space weren’t being filtered properly when deciding whether they should contribute to the heatmap. The issue was this: if a room had sensors of the type being displayed, then all the room’s sensors would be added to the list to display. This wasn’t an issue for the Hyperion reference application, interestingly, as they only have sensors with multiple sensor feeds (what we call equipment sensors in Dasher-speak). But in Dasher we often have a varied set of sensors in a room, and they should not all be contributing data to planar heatmaps. Anyway, this was another important fix for us, as otherwise the heatmaps were rather saturated and – most importantly – displayed the data incorrectly.
The Hyperion team also had to mitigate for some issues that were in the underlying Heatmap.js library – which is being used to display heatmaps in a plane. A really interesting issue was that when data was exactly at the minimum of the range (when normalised this means it’s 0 in the 0.0 to 1.0 range) then the value would be displayed as the maximum (1.0)! This was a real surprise to me – and I still don’t understand the basis or reason for this – but it was only evident when dealing with sensors such as presence sensors, which typically deal with more discrete values (1.0 if someone’s in the space, otherwise 0.0 if it’s empty). When dealing with temperature/CO2/humidity/etc. you would rarely see a value at exactly the bottom of the range. The fix was easy on this one: just passing zero values as tiny decimals (0.0001) meant the data was displayed properly (or much more properly, anyway).
Another important addition related to minimum and maximum opacity. By default Hyperion’s planar heatmaps have colours/values lower in the range be displayed increasingly transparently. This makes sense when you’re using a heatmap to display coverage or concentration: when dealing with sensor data you typically want the minimum data to be shown just as opaquely as the maximum data. The Hyperion team kindly exposed a minOpacity option (which was already supported by Heatmap.js) to allow developers to specify that opacity should not vary based on the value of the data (by setting this option to 1). Here’s an example of how different the data looks with this setting on or off:
One more issue with data representation related to the display of NaN values. (Remember, everyone – you need to take great care of your NaNs… ;-) In JavaScript NaN stands for “Not a Number”, and is a common-place occurrence in the real world of sensor data. Sometimes the sensor generates data that is for whatever reason invalid, and this reading might very well get stored in (or served up by) the time-series database as a NaN. So when displaying sensor data it’s important not to assume it’s going to be within a particular range – it could well be an outlier or a NaN.
In prior versions of Hyperion, planar heatmaps would display NaN values at the minimum of the colour range (which, as described earlier in the post, would actually lead to the sensor being showing to be at the maximum of the range!) but even setting this to the minimum is incorrect (although it’s not awful when the minOpacity is set to 0, as it would then not be shown): it’s important to exclude the display of NaN values completely from the display of heatmaps. They should not contribute in any way (sorry, NaNs!). This is the way the volumetric shader worked already (this is something we paid close attention to addressing in Dasher from its early days) and I’m happy to say that planar heatmaps now work similarly.
Earlier in the post I said we’d talk a little about placement of the heatmaps: currently it’s possible to place them at the top or the bottom of the 3D space’s bounding box. This might – depending on how your rooms have been modelled – allow you to display your data on the floor of your space (which is in many ways ideal), but this doesn’t currently work for Dasher: we have to have the room volumes modelled to contain floor-/ceiling-mounted sensors, so the room volumes are typically larger than the internal spaces defined by the walls/floors/ceilings. Which also means that showing heatmaps at the minimum will lead to them being hidden beneath flooring. Showing at the maximum does work, so we’re gone with this option for now. (The Hyperion team has offered to support this being defined as a 0 to 1 value – so setting placement at 0.1 would mean it would be 10% above the bottom of the space – which we’ll almost certainly take advantage of, but that is something that’s likely to come in an upcoming release.)
I also said I’d talk a little about why planar heatmaps often don’t work for sensor data as well as volumetric ones. Let’s take the example of this for temperature data… in one part of the floor – the lower left, in this view – we have a set of sensors in adjacent cabins. Despite being in separate, closed offices, as they’re located closely to one another they contribute to making that area of the planar heatmap much more red than it should be: each sensor is reporting 24.5-25 degrees Celsius, while the perceived value – based on the colour display – is up at around the 28 degrees. This area looks just as warm as the room towards the right – which is surrounded by glass, and is very often up at 28 degrees in Summer – which only has a single sensor. So you do need to think carefully about how planar heatmaps are used, and whether they make sense for your data.
If you want to try planar heatmaps for yourself, I’ve now pushed the implementation to dasher360.com. You can enable the feature via the Appearance tab in Dasher’s settings:
A huge thanks to the Hyperion team for their hard work to make the v7.44 release of the viewer work so well for us. While it was definitely a team effort on their side, it’s worth drawing attention to the contribution by my old friend Beng Lin Goh, who spent countless late nights and even some time on weekends responding to my niggling feedback. Thank you, Ben and team!
Next on the list for Hyperion exploration is per-object heatmap display. We’ve used this, in the past, to display heatmaps on the surface of the MX3D bridge, but we’ll also look at how you can use this capability to shade geometry such as sensor objects based on their value.