Now that we’ve seen how to move a fake cursor across the screen, let’s talk about how best to use this technique to implement a demo mode for your Forge viewer-based application.
Something I mentioned last time was that – for our purposes – this code needs to be fairly adaptable, whether for different views, models or screen configurations. It also needs to be easy for us to modify or extend.
Inside Dasher 360 we’ve implemented a number of different extensions. Many of these have their own UI, typically launched by a button. One of the first things I did for this project was to make each extension able to report the location of its launch button: this way we can query the information and use it to move the cursor to that location in order to simulate the launch action (we load and run the extension from our code, of course).
Here’s a snippet of TypeScript code that shows how this works, more or less (there are dependent functions that I haven’t included):
clickOnExtensionButton(ext: ShowableExtension, show = true): Promise<any> {
return new Promise((resolve, reject) => {
let loc = ext.buttonLocation;
this.moveCursor(loc.x, loc.y).then(() => {
ext.hoverOverButton(true);
ext.showTooltip(true);
setTimeout(() => {
ext.showTooltip(false);
}, this._pauseToHighlight * 2);
}).then(() =>
this.pause(this._pauseToHighlight)
).then(() => {
if (show) {
ext.show();
} else {
ext.hide();
}
ext.setButtonActive(show);
}).then(() =>
this.pause(this._pauseToHighlight)
).then(() => {
ext.hoverOverButton(false);
resolve();
});
});
}
Each extension can also register “interesting locations”, where the cursor should also visit during the demo. These locations aren’t just the points on the screen to which the cursor should travel, but the actions that need to be performed both when the cursor hovers over the location and when it “clicks” on it.
I won’t go into the specifics of how this was implemented, but here’s the overall flow of the demo. This below TypeScript code should give a good idea of the sequence of operations and how composable/editable the demo becomes.
performDemo(): Promise<any> {
return new Promise<any>((resolve, reject) => {
let nodes = this._dataModel.sensorNodes;
let info = nodes.map(node => this._dataModel.getSensorInfo(node));
let connected = info.filter(inf => inf.isInDB);
let ids = connected.map<[string, number, THREE.Vector3]>(inf => [inf.id, inf.nodeid, inf.location]);
let random = this.nRandomElements(ids, 5);
// Start by clicking on the sensor list and sensor dots extension buttons,
// along with the "interesting locations" for the sensor list (in our case
// this means the "hide unconnected" button)
let fewer: [string, number, THREE.Vector3][];
Promise.resolve().then(() =>
this.resetDemo() // Reset some things to make the next iteration cleaner
).then(() =>
this.clickOnExtensionLocations(this._bcrumbsExt, 1) // Click on the first breadcrumb location
).then(() =>
this.clickOnExtensionButton(this._senListExt)
).then(() =>
this.clickOnExtensionButton(this._senDotExt)
).then(() =>
this.clickOnExtensionLocations(this._senListExt)
).then(() =>
fewer = random.filter(id => this.isPointBehindUIPanel(this.getSensorLocation(id[1], id[2]))) // Filter any points that will be behind UI elements
).then(() =>
this.hoverOverSensorsClickOnLast(fewer) // Hover over a random sampling of connected sensors
).then(() =>
this.clickOnExtensionButton(this._senListExt, false) // Close the sensor list
).then(() =>
this.clickOnExtensionButton(this._dashExt) // Launch the dashboard
).then(() =>
this.clickOnLocations(this.nRandomElements(this._dashExt.kioskSensorLocations, 2)) // Click on a couple of random dashboard sensors
).then(() =>
this.clickOnExtensionButton(this._plotExt, false) // Close the plots
).then(() =>
this.clickOnDashboardTab('navigation') // Click on the navigation dashboard tab
).then(() =>
this.clickOnLocations(this.nRandomElements(this._dashExt.kioskNavLocations, 2)) // Click on a couple of random navigation bookmarks
).then(() =>
this.clickOnExtensionButton(this._dashExt, false) // Close the dashboard
).then(() =>
this.clickOnExtensionLocations(this._bcrumbsExt) // Click on breadcrumbs
).then(() =>
this.clickOnExtensionButton(this._senDotExt, false) // Turn off sensors
).then(() =>
this.clickOnExtensionButton(this._textExt) // Launch the texture hiding extension
).then(() =>
this.clickOnExtensionButton(this._surfShadExt) // Launch the surface shading
).then(() =>
this.pause(this._pauseToHighlight * 4) // Pause before resetting
).then(() =>
this.clickOnExtensionButton(this._surfShadExt, false) // Turn off surface shading
).then(() =>
this.clickOnExtensionButton(this._textExt, false) // Turn off texture hiding
).then(() =>
resolve()
);
});
}
Here’s a video of the demo in action. Remember: this is a recording of the browser but the only manual UI event was when I clicked on the button to launch “Kiosk mode”. The rest is the browser scripting (and simulating) various operations.
What’s nice is that a number of the operations have random elements. This means the demo can repeat but will have different results each time (irrespective of whether new sensor data has been received).
In the next post we’ll take a look at the approach taken to make the loop repeat endlessly, stopping only when the mouse is moved.
