Implementing a demo mode inside the Forge viewer Part 1

A request came up in a recent meeting talking about future features for Dasher 360. We want to be able to leave Dasher 360 running in “kiosk mode”, whereby if left unattended for some time it starts to run through some canned activities, showing off some of the tool’s capabilities. The mode should be interruptable: when the mouse moves the mode should stop, allowing the user to continue exploring on their own.

This sounded like a really fun problem to tackle, so I started on it right away. My first avenue of research was around tools that already exist for capturing/playing back mouse activity in web-pages. The kind of thing you might use for automated testing, for instance.

Very early on I realised that browsers don’t have this capability built-in: for lots of very good reasons you can’t control the mouse from within a web-page (e.g. via JavaScript). This is something we take for granted in the desktop world, but it’s just not available in the browser sandbox.

That said, you can use an interesting (and slightly scary) technique called cursor spoofing or cursorjacking. Which basically means you hide the cursor and display your own in its place (or rather in the place you want it). When used maliciously in web-sites this usually tracks relative to the existing cursor, tricking the user into clicking on things they shouldn’t. In our case we just want to move the cursor around and simulate using certain UI features.

Some work is going to be needed from the features themselves to make them scriptable/clickable by a fake cursor. The goal is to implement this in a way that is:

Easy to modify/extend to new features
View-independent
Not tied to a specific model
Functional for any screen configuration/resolution

Which all means avoiding classic efforts to capture cursor movements/clicks and replay them back. We need to determine areas of interest based on 3D geometry as well as the UI that’s been added to the Forge viewer – ideally asking the various extensions to define the areas of interest and what happens when you hover over or click on them – and define some infrastructure to run the various operations in a loop.

Sounds like fun, right? It did to me, too. :-)

So, starting at the beginning, let’s look at some TypeScript code that spoofs the cursor – making the original invisible and displaying a fake one – and then scripts its movement from one place to another on the screen.

Here’s how we hide the cursor – both for the Forge viewer’s canvas and for the UI elements we’ve added.

// We need to hide the cursors for both the canvas and the UI elements (toolbar buttons).
// Store the values so that we can restore them at the end
 
this._savedCursor = this.viewer.canvas.style.cursor;
this.viewer.canvas.style.cursor = 'none';
let buttons = $('.adsk-button');
this._buttonCursor = buttons[0].style.cursor;
buttons.css('cursor', 'none');
 

Here’s the code to dynamically create the cursor in the DOM; _cursorId and _cursorUrl are member variables that refer to what we’re calling our element and the image file representing the fake cursor.

if (!$('#' + this._cursorId)[0]) {
  let $cursor = $('<img src="' + this._cursorUrl + '" id="' + this._cursorId + '" />');
  $('body').append($cursor);
}
 

And here’s a function that moves the fake cursor from its current location (stored in _canvasX and _canvasY, which is set by a ‘mousemove’ handler) to a new location:

moveCursor(x: number, y: number, addHeader?: boolean): Promise<any> {
 
  return new Promise((resolve, reject) => {
    if (this._inKioskMode) {
      if (addHeader) {
        y = y + 51; // Our navigation header is 51 pixels high
      }
      let img = $('#' + this._cursorId);
      img.css('display', 'block');
      img.css('position', 'absolute');
      img.css('z-index', 9999);
 
      // The Kiosk button is approximately 70% down the screen
 
      let startX = this._canvasX || 25;
      let startY = this._canvasY || window.innerHeight * 0.7;
 
      // Base the number of steps on the distance to travel
      // (i.e. approximate a constant speed)
 
      let xdiff = x - startX;
      let ydiff = y - startY;
      let dist = Math.sqrt(xdiff * xdiff + ydiff * ydiff);
      let steps = Math.floor(dist / 3);
      let incX = xdiff / steps;
      let incY = ydiff / steps;
 
      let pos = 0;
      let id = setInterval(() => {
        if (pos === steps) {
          clearInterval(id);
          resolve();
        } else {
          pos++;
          let newX = startX + (pos * incX);
          let newY = startY + (pos * incY);
          img.css('left', newX + 'px');
          img.css('top', newY + 'px');
          this._canvasX = newX;
          this._canvasY = newY;
        }
      }, 10);
    }
  });
}
 

You’ll notice that the code returns a Promise – a piece of asynchronous code that can execute pseudo-synchronously – so that we can chain this operation together within a sequence.

Let’s take a look at a sample piece of the sequence – here we get the screen location of a particular sensor (yes, hardcoded for testing purposes) in our model and then move the cursor to it.

Promise.resolve().then(() => {

let loc = this._dataModel.getSensorLocation(2338);

let pt = this.viewer.worldToClient(loc);

this.moveCursor(pt.x, pt.y, true);

}).then(() =>

...

)

Here’s the code in action:

The beauty of this is this approach is that if we change the viewpoint it still moves the cursor to the same sensor. Which opens all kinds of possibilities.